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ABSTRACT 


The  optimization  problem  as  formulated  in  the  METRIC  model  lakes  llie 
form  of  minimizing  the  expected  number  of  total  system  backorders  in  a two- 
echelon  inventory  system  subject  to  a budget  constraint.  The  system  contains 
recoverable  items  - items  subject  to  repair  when  they  fail.  To  solve  this  prob- 
lem, one  needs  to  find  the  optimal  Lagrangian  multiplier  associated  with  the 
given  budget  constraint. 

Kor  any  large-scale  inventory  system,  this  task  is  computationally  not  trivial. 
Fox  and  Landi  proposed  one  method  that  was  a significant  improvement  over 
the  original  METRIC  algorithm.  In  this  report  we  first  develop  a method  for 
estimating  the  value  of  the  optimal  Lagrangian  multiplier  used  in  the  Fox-Landi 
algorithm,  present  alternative  ways  for  determining  slock  levels,  and  compare 
these  proposed  approaches  with  the  Fox-Landi  algorithm,  using  two  hypotheti- 
cal inventory  systems  - one  having  3 bases  and  75  items,  the  other  5 bases 
and  125  items.  The  comparison  shows  that  the  computational  time  can  be  re- 
duced by  nearly  50  percent 

Another  factor  that  contributes  to  the  higher  requirement  for  computational 
lime  in  obtaining  the  solution  to  two-echelon  inventory  systems  is  that  it  has  to 
allocate  stock  optimally  to  the  depot  as  well  as  to  bases  for  a given  total-system 
slock  level  This  essentially  requires  the  evaluation  of  every  possible  combina- 
tion of  depot  and  base  slock  levels  — a time-consuming  process  for  many  prac- 
tical inventory  problems  with  a sizable  system  slock  level  This  report  also  sug- 
gests a simple  approximation  method  for  estimating  the  optimal  depot  slock 
level  When  this  method  was  applied  to  the  same  two  hypothetical  inventory 
systems  indicated  above,  it  was  found  that  the  estimate  of  optimal  depot  stock 
is  quite  close  to  the  optimal  value  in  all  cases.  Furthermore,  the  increase  in  ex- 
pected system  backorders  using  the  estimated  depot  stock  levels  rather  than  the 
optimal  levels  is  generally  small 


I.  INTRODUCTION 


Almost  a decade  ago,  Sherbrooke  formulated  the  well-known  METRIC  model  for  deter- 
mining optimal  stock  levels  for  recoverable  items  — items  subject  to  repair  when  they  fail  — in 
a two-echelon  setting  [3],  Briefly,  the  two-echelon  system  consists  of  several  locations,  called 
bases,  at  which  primary  demands  occur;  these  bases  are  in  turn  resupplied  as  neressaiy  by  a 
central  repair  and  inventory-stocking  point  called  a depot.  When  a failure  occurs  at  a base,  a 
demand  is  placed  on  base  supply  for  a corresponding  replacement  part.  The  failed  part  is  then 
either  repaired  at  that  base,  or  is  sent  to  the  depot  for  repair,  depending  on  the  nature  of  the 

•This  research  was  partially  supported  by  Ihe  Office  of  Naval  Research  under  Conlraci  NOOOI4-75-C-1 172. 

Task  NR  042-335,  and  by  the  RAND  Corporation. 
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failure.  Resupply  of  base  supply  comes  from  the  base  maintenance  organization  if  repair  is 
accomplished  at  the  base;  otherwise,  resupply  comes  from  the  depot.  In  either  case,  the  organi- 
zation resupplying  the  base  supply  activity  does  so  by  exchanging  a serviceable  part  for  the 
failed  part.  Thus  the  inventory  policy  for  placing  orders  on  the  base’s  maintenance  organization 
or  the  depot  is  an  (s  - l.s)  policy. 

Sherbrooke  presented  a model  that  can  be  used  to  determine  both  depot  and  base  stock 
levels  for  all  items  for  this  system.  In  particular,  the  problem  he  formulated  was  that  of 
minimizing  the  expected  total  number  of  base  back  orders  existing  at  an  arbitrary  time  subject 
to  a constraint  on  system  investment,  that  is, 

min  £ ]£  £ (x  - s/y)pU|A/y7’/(s<0)] 

J-\  x>itj 

(PI)  subject  to  £ £ cis»j  < c- 
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expected  number  of  units  in  the  resupply  system  is  y. 


Furthermore,  Sherbrodke  shows  that  Tyis,J  can  be  expressed  as 

T/yls.o)  ■ t rj  + (1  — t jj)  [ 3jj  + d(s,(j)  ■ Dj, 

where 


^ u 
Ta 

Bij 

a(s,o)  ■ A 


the  average  base  repair  time  for  item  / at  base  j, 

the  proportion  of  demands  requiring  base  repair  for  item  i 

at  base  y, 

the  average  order-and-ship  time  at  base  j for  item  /, 
the  average  depot  repair-cycle  time  for  item  i, 
expected  depot  backorders/expected  depot  daily  demand  rate 
(1/A/)  £ (x  - j/o)p(x|X/D/),  the  expected  delay  per 
*>«„ 

depot  demand  for  item  i,  and 

m 

£ (1  - r,j) X//,  the  expected  daily  depot  demand  for  item  I. 
y-i 


In  the  remainder  of  the  report,  / will  refer  to  an  item  and  J will  refer  to  a base  (j  — 0 represents 
the  depot).  For  a complete  description  of  the  problem's  background  and  formulation,  see  Ref. 
13]. 


Subsequently,  Fox  and  Landi  suggested  a Lagrangian  approach  for  solving  problem  PI  [2], 
One  obstacle  to  the  implementation  of  METRIC  using  the  Fox-Landi  algorithm  is  the  require- 
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ment  of  estimating  an  appropriate  value  for  the  Lagrangian  multiplier.  Another  important  and 
related  problem  is  the  lengthy  computer  run  time  required  to  obtain  an  optimal  solution  to 
problem  PI  when  using  their  algorithm.  A large  portion  of  this  computational  effort  is  related 
to  searching  for  the  optimal  depot  stock  level.  This  search  is  particularly  time-consuming  for 
items  having  a high  average  number  of  units  in  the  depot  repair  cycle  since  the  amount  of  com- 
putation required  by  their  algorithm  is  roughly  proportional  to  the  number  of  depot  stock  levels 
explicitly  examined. 

In  this  paper,  we  first  develop  an  approach  for  obtaining  an  estimate  of  the  optimal 
Lagrange  multiplier  value  required  in  the  Fox-Landi  algorithm,  present  two  new  methods  for 
determining  stock  levels,  and  compare  these  methods  with  the  Fox-Landi  method  and  other 
techniques.  The  proposed  approach  eliminates  the  particularly  time-consuming  portion  of  the 
Fox-Landi  algorithm  devoted  to  searching  for  the  best  J agrange-multiplier  value  and 
significantly  reduces  computation  time  for  determining  stock  levels  without  degrading  the  qual- 
ity of  the  solution. 

We  then  present  a method  for  estimating  the  optimal  depot  stock  level.  Limited  compu- 
tational experience  indicates  that  this  method  is  easy  to  implement,  provides  a very  good  esti- 
mate of  the  optimal  depot  stock  level,  and  is  particularly  useful  for  items  having  a high  average 
number  of  units  in  the  depot  repair  cycle.  For  these  items  it  is  possible  to  reduce  computation 
time  required  by  the  Fox-Landi  algorithm  by  as  much  as  90  percent. 

II.  THE  APPROXIMATION  PROBLEM 

In  this  section,  we  first  construct  a problem  that  is  a continuous  approximation  to  problem 
PI.  We  then  state  and  prove  two  theorems  that  are  the  basis  for  an  algorithm  that  can  be  used 
to  solve  this  approximating  problem. 

Two  useful  probability  distributions  for  describing  the  demand  process  are  the  Poisson 
and  negative-binomial  distributions.  As  shown  in  Ref.  [1),  this  implies  that  if  demand  has  a 
Poisson  or  a negative-binomial  distribution,  then,  for  a given  value  of  X„T,/(s,0),  the  probability 
distribution  representing  the  number  of  units  in  resupply  of  items  / at  base  j at  any  time, 
/(*,<>)]>  is  a Poisson  or  a negative-binomial  distribution,  respectively. 

Experimental  data  gathered  during  the  conduct  of  this  study  indicate  that,  when 
plx|X„7',;(j,o)l  is  either  a Poisson  or  a negative-binomial  distribution,  the  expected  number  of 
back  orders  at  each  location  can  be  closely  approximated  by  an  exponential  function.  This  is 
not  unexpected.  First,  for  budgets  of  practical  interest,  the  item  stock  levels,  s„,  are  normally 
much  larger  than  the  average  demand  during  the  resupply  time.  In  fact,  the  probability  of  run- 
ning out  of  stock  during  the  resupply  time  is  often  much  less  than  0.1  S in  real  applications. 
Thus,  the  only  probabilities  entering  the  backorder  calculation  are  the  tail  probabilities  of  the 
distribution.  In  the  tails,  both  the  Poisson  and  negative-binomial  distributions  behave  almost 
like  the  geometric  distribution;  that  is,  each  succeeding  probability  is  roughly  a constant  propor- 
tion of  its  predecessor.  Consequently,  when  s„  is  large  relative  to  W„7',/(i,0),  the  expected 
number  of  backorders  existing  at  any  time  at  location  j for  item  i is  approximately  a geometric 
function  of  s„.  Therefore,  an  exponential  function  is  a useful  continuous  approximation  to  this 
relationship  between  expected  backorders  at  a location  and  the  itemls  stock  level  at  that  loca- 
tion. 


Furthermore,  total  expected  base  backorders  for  each  item  exhibit  this  same  behavior.  If 
demand  has  either  a Poisson  or  a negative-binomial  distribution  (or,  for  that  matter,  any  com- 
pound Poisson  distribution),  then  the  total  number  of  units  of  an  item  in  resupply  across  all 
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bases  has  either  a Poisson  or  a negative-binomial  distribution,  respectively,  if  we  assume 
independence  of  demand  and  common  variance-to-mean  ratio  among  base  demand  distribu- 
tions. Since,  in  most  practical  situations,  total  system  stock  substantially  exceeds  the  total 
expected  number  of  units  in  resupply,  the  tail  of  the  distribution  describing  the  total  number  of 
units  in  resupply  is  the  only  portion  of  the  distribution  of  importance.  As  an  approximation, 
this  distribution  can  be  used  to  determine  the  nature  of  the  relationship  between  total  expected 
base  backorders  and  total  system  stock.  For  the  reasons  discussed  previously,  an  exponential 
function  should  also  adequately  represent  this  relationship. 

Thus  we  will  approximate 

m 

I I (jt-  s^pUlx^r.jfs^l 

/-!  <r>v„ 

with  the  exponential  function 

B,(N)  = a,e~h'N‘. 

In  this  approximation,  N,  represents  total  system  stock.  In  practice,  the  parameters  a,  > 0 and 
b,  > 0 are  estimated  using  regression  analysis.  The  data  used  in  the  regression  analysis  are  the 
backorder  data  obtained  from  the  solution  to  the  problem 

min  £ £ (x  - j(/)p[x|Xj(r„(j,o)] 

i- 1 

m 

subject  to  £ s„  - N,  and 

(-0 

»„-  0,1 N„ 

for  several  appropriate  values  of  N,. 

We  now  formulate  a continuous  approximation  to  problem  PI  in  which  the  exponential 
approximation  to  total  system  backorders  for  an  item  is  used.  In  this  approximation  problem, 
the  decision  variables  are  the  total  system  stock,  N„  rather  than  the  stock  levels  for  each  loca- 
tion, s,).  This  approximation  problem  is  a vehicle  for  obtaining  an  estimate  of  the  optimal 
Lagrangian-multiplier  value  used  in  the  Fox-Landi  algorithm.  The  approximation  problem  is 
formulated  as  problem  P2: 

min  £ 

/-I 

(P2)  subject  to  £ c,/V,  < C, 

i-i 

where 

N,  > 0. 

Note  that  N , is  a continuous  variable  in  this  approximation.  The  optimality  (Kuhn-Tucker) 
conditions  for  this  problem  are  as  follows: 


Find  0,  > 0 such  that 
(a) 


dB, 

— + 0{c,  > 0 , 
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(b) 

N,>  0, 


II 


(c) 

9\  | 

£c,A,  - C 

j = 0,  and 

(d) 

N, 

dB, 

4*  0 \ c. 

= 0. 

dN,  1 ' 

A relaxed  version  of  problem  P2  in  which  the  nonnegativity  constraint  on  the  item  stock 
level  is  removed  is  problem  P3: 

min  £ fl,(  A,) 

i-i 

(P3)  subject  to  £ c,A , < C. 

i-i 

The  optimality  conditions  for  this  problem  are  similar  to  those  for  P2.  In  condition  (a)  the  in- 
equality is  replaced  by  an  equality  and  the  nonnegativity  restriction  on  A,  is  omitted.  Also  02 
will  represent  the  Lagrangian  multiplier  for  problem  P3. 

We  now  explore  the  relationship  between  problems  P2  and  P3  in  detail. 

Suppose  we  solve  problem  P3.*  Let  A,1  represent  the  optimal  solution  to  problem  P2, 
and  A,2  represent  the  optimal  solution  to  problem  P3.  If  A,2  ^ 0 for  all  /',  then  A,1  =■  A,2  and 
the  objective  function  values  are  equal. 

If  A,2  < 0 for  at  least  one  value  of  /,  let 

A,  - max(0,  A,2) 
and 


C = £ r,A,  > C. 

<-i 


Suppose  problem  P2  is  modified  slightly  so  that  C is  replaced  by  C.  This  modified  prob- 
lem is  called  problem  P4.  The_optimality  conditions  for  this  problem  are  the  same  as  those  for 
problem  P2  after  substituting  C for  C.  Also,  let  0 represent  the  optimal  value  of  the  Lagran- 
gian multiplier  for  problem  P4. 

2n  solving  problem  P3,  we  will  obtain  a value  for  92.  It  is  easy  to  show  that  0 - 02  and 
that  A,  - max(0,  A,2)  is  an  optimal  solution  to  problem  P4  by  demonstrating  that  these  values 
satisfy  the  Kuhn-Tucker  conditions  corresponding  to  problem  P4.  Consequently,  the  optimal 
solution  to  problem  P4  is  A,  - A,  - max(0.  A,2).  Furthermore,  the  optimality  conditions  are 
satisfied  when  9 is  equal  to  9 2. 

THEOREM  1:  0,  > 02  - 9. 


'Scclion  III  develops  the  method  for  determining  ihc  solution  lo  problem  P3 
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PROOF:  The  optimal  objective  function  value  for  problem  P2  is  a convex,  differentiable, 
strictly  decreasing  function  of  the  available  budget,  C.  Since  the  slope  of  thjs  function  is  equal 
to  the  negative  of  the  Lagrangian-multiplier  value,  0,  > 0 since  C < C.  But  — 0,  so 
0,  ^ 02- 

Nexi_we  compare_A^,1  with  N,.  Suppose  C > C so  that  0,  > 02  = 0.  Let  us  examine  the 
two  cases  /V,  > 0 and  N,  = 0 separably. 

First,  assume  N,  > 0.  Then 

+ 0c,  = 0. 

N-N, 

Furthermore,  if  N ,'  > 0,  then 


dB, 

'dN, 


dB, 

~dN, 


+ 0,c,  = 0. 


Since 


•»c'  > ~9c'  " - ik, 

dB, 


N-N 


dB, 

In', 


n-n,  dN, 


N -N  1 


and  N ,’  < N,.  If  W,1  = 0,  then  N,  > N,\ 


Next,  assume  N,  — 0.  Since 

+ 0c,  > 0, 

N-  0 

it  follows  that  /V,1  - 0 by  complementary  slackness.  Thus  we  have  proven  the  following 
theorem. 


dB, 

dN, 


dB, 

n.«  + 9iCi>  m 


THEOREM  2.  N,  > N,'  and  N,  > N,'  whenever  N,  > 0. 


Having  established  several  important  relationships  among  problems  P2,  P3,  and  P4,  we 
next  develop  a simple  algorithm  for  solving  problem  P2  and  show  how  to  find  the  solution  to 
problem  P3. 

III.  COMPUTING  OPTIMAL  SOLUTIONS  FOR  PROBLEMS  P2  AND  P3 


Observe  that  the  optimal  solution  to  problem  P3  must  satisfy  the  two  conditions 


dB, 


and 


I c,N,  - C 

/-I 


because  each  B,(N ,)  is  a strictly  decreasing  function  of  N,. 
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Since 


B,(N,)  = a,e~h'N‘. 


where  u„  b,  > 0,  the  first  condition  states  that 


. 

a,b,e 

c, 


> 0. 


Letting 


and 


we  see  that 


9 A In  0,  = In 


d,  In 


- 


o,6, 


(V, 


d,  - 9 


From  the  second  constraint  we  know  that 

d,  - 9 


Thus 


Letting 

we  can  express  9 as 

Thus 

(El) 

and 

(E2) 


/-i 


L (c,d,/b,)  - c 

. 

Z (c,/6,) 

/-I 

" ci^i  " c, 

« - X — and  p = £ 

<-i  ,-i 


a - C 


0,  _ eia-C)/fi 


d,-z^c 

ft,-—,  p 


Consequently,  N,  is  a linear  function  of  C. 


... 
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We  may  employ  the  following  algorithm  to  find  the  optimal  solution  to  problem  P2.  Let 
/ - (1 «}  and  A',1  represent  the  optimal  solution  to  problem  P2. 

STEP  0:  Solve  Problem  P3  as  described  above,  thereby  obtaining  an  initial  value  for 

N„  i € /. 


STEP  1:  Set  N,'  - 0 for  all  N,  < 0 during  the  last  iteration  and  delete  the  corresponding  / 
from  /.  Recompute  a and  0,  where 


and 


a 


0-1 

iii 


i 


STEP  2:  Using  (E2),  obtain  new  estimates  of  N,  for  each  / € /.  If  N ^ 0 for  all  / € /,  ; 

then  the  optimal  solution  has  been  found,  and  N - N,  for  all  / 6 / and  /V1  = 0 for  all 
/-l n for  which  / i I.  If  there  exists  some  i for  which  N,  < 0,  return  to  Step  I. 


It  is  clear  that  our  solution  satisfies  all  the  optimality  conditions  for  problem  P2  except, 
possibly,  condition  (a)  for  / / /.  However,  at  an  earlier  iteration  (when  i was  deleted  from  / ) 
we  had 


dB, 

dN, 


. + »2C, 


0, 


where  f)2  ar>d  2V, ( < 0)  are  earlier  values  of  02  and  respectively.  Since  dBJdN,  is  clearly 
increasing  in  N„  and  02  increases  at  each  iteration  (Theorem  I and  its  corollary),  condition  (a) 
must  hold.  Convergence  is  guaranteed  since  n is  finite. 


IV.  A COMPARISON  OF  ALTERNATIVE  SOLUTION  PROCEDURES 
FOR  SOLVING  PROBLEM  PI 


In  this  section  we  review  three  algorithms  for  solving  problem  PI  and  compare  them  to 
two  algorithms  designed  to  obtain  a solution  for  problem  PI  based  on  the  solution  to  the 
approximating  problem,  problem  P2. 

The  Sherbrooke  Procedure 


The  first  algorithm,  a procedure  originally  proposed  by  Sherbrooke  [31,  is  a marginal- 
analysis  algorithm  consisting  of  two  phases.  In  the  first  phase,  each  item  is  examined  indepen- 
dently. The  optimization  problem  solved  for  item  / in  the  first  phase  has  the  form: 

W 

Z,(N,)  — min  £ £ (jt  — s,,)/z [jc ( A ,, 7',,(s,0)  1 

/-I  »>.,( 

m 

(P5)  subject  to  ^ s„  - A',, 

i-0 

where 

0.  1 
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and  N,  is  the  total  system  stock  available  for  distribution  among  the  depot  and  bases.  Problem 
P5  is  solved  by  obtaining  the  solution  to  the  N,  + 1 problems 

2,{N„  s(0)-min  £ £ (jc  - s(/)^ [jc | X(/ 7'J/(jr,0)] 

/-I  v>vf/ 

<P6)  subject  to  £ s„  - N,  - s,0, 

i-\ 

where 


*«-0.  1 

and  s,0  is  fixed  for  s,0  - 0,  1,  ....  N,.  Problem  P6  can  be  solved  via  marginal  analysis.  Then 

Z,(N)  - min  Z,(/V„  s(0), 

Vo 


where 


s,o  “ 0. 


N,. 


The  second-phase  problem  is 


where 


min  £z,(/V,) 

n 

subject  to  £ c<ty  < C, 

>-i 


N,  - 0.  1 

Sherbrooke  [3]  suggests  that  a marginal-analysis  algorithm  be  used  to  find  a solution  to  this 
knapsack  problem.  Clearly  other  procedures  could  be  employed  to  obtain  an  optimal  solution. 
In  any  case,  this  approach  requires  a substantial  amount  of  storage  to  save  all  the  Z,(/V,)  values. 
For  moderate-sized  problems  having  several  thousand  items,  a storage  requirement  of  106  or 
more  words  may  be  needed  to  save  these  values.  Furthermore,  the  computation  time  required 
to  obtain  these  Z,(N)  values  for  such  problems  is  very  large. 

The  Fox-Landl  Procedure 

Subsequently  Fox  and  Landi  [2]  proposed  a Lagrangian  algorithm  for  solving  problem  PI. 
In  particular,  they  formulated  the  relaxed  version  of  problem  PI  as  problem  P7: 

(p7)  niin  £ £ £ (x  - s„)/»[x|x(/7'(/(s/0)l  + 9 £ £c,s//( 

/-I  i-l  »>»,,  ,-0  i-l 

where 

in -0.  1 

and  9 ia  the  Lagrangian  multiplier.  Since  problem  P7  is  separable  by  item,  its  optimal  solution 
can  be  found  by  solving  the  n individual  item  problems 

min  £ £ (x  - + 9 £c,j„ 

/-I  <r>t„  /- 0 


subject  to 


1 
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This  problem,  like  problem  P6  in  Sherbrooke’s  two-phase  method,  is  solved  using  a partitioning 
procedure,  that  is,  it  is  reformulated  as 


(P8) 

min 

m 

®c.-s.o  + X min 

X,  (*  - s,/)p[x-|x,/7’„(s,0)]  + 9c,s,,:  s,0  fixed 

\0“0. 1. ... 

or  equivalently  as 

(P9)  min  Z(sl0;  9) 


where 

s,o  - 0,  1 

and 


Z (s,0;  9)  = 9c,s,0  + X min  5-  " s,/)p[*|A(,7’„-(s,o)] 

;-l  *//  x >s0 

+ 9c,s,,:  s(/  — 0,  1 , . . . ; s,0  fixed  , 

To  determine  Z(s(0;  9 ),  solve  the  m base  problems 

min  £ (x  - s„)p[x|A/(7’,((s,0)]  + 9c,s,r 

*1/  V > Sjj 

The  optimal  s„  is  the  smallest  nonnegative  integer  for  which 

X pU|A.,/7’,/(s,0)]  9c,. 

*>',! 

Problem  P8  is  solved  for  each  item  for  a given  value  of  9.  This  yields  a total  investment 
cost  corresponding  to  9.  In  the  Fox-Landi  approach,  the  "optimal"  value  of  9 is  selected  from  a 
grid  of  M equally  spaced  values, 

90  > 0,  > ...  > 9m  > 0. 

The  optimal  value  of  9 is  the  9 K,  K € |0,  ...  ,A/),  whose  corresponding  total  investment  cost  is 
closest  to  C. 

Fox  and  Landi  suggest  that  their  method  is  a single-pass  method;  that  is,  only  one  pass 
through  the  item  data  base  is  necessary  to  obtain  the  optimal  solution.  The  storage  require- 
ment to  effect  this  one-pass  approach  is  potentially  enormous.  For  a moderate-sized  problem 
having  3000  items,  20  bases,  and  M — 63,  almost  4 million  item  stock  levels  must  be  saved, 
plus  possibly  millions  of  additional  item  data  elements  reflecting  fill  rates,  probability  of  no 
stockout  at  an  arbitrary  time,  expected  base  backorders,  and  so  on.  Furthermore,  because  there 
may  be  no  simple  method  for  estimating  suitable  bounds  on  the  values  of  the  multipliers,  much 
larger  values  of  M may  be  required  to  ensure  adequate  approximation  of  the  budget. 

It  has  been  the  author's  experience  that  Air  Force  personnel  have  difficul  ; estimating  a 
reasonable  range  for  9 for  large  problems.  This  is  not  surprising,  because  the  data  used  in  the 
model  frequently  change  in  real  situations,  thereby  causing  the  optimal  value  of  the  multiplier 
to  change.  Furthermore,  changing  the  multiplier’s  magnitude  by  10~6  or  less  often  causes  the 
corresponding  total  cost  to  change  by  many  millions  of  dollars.  Consequently,  2 10  values  of  9 
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have  been  used  in  some  Air  Force  applications  to  make  the  system  "foolproof."  In  these  cases, 
60  million  or  more  item  stock  levels  would  have  to  be  explicitly  stored,  plus  a considerable 
amount  of  other  item  and  base  data,  to  make  the  Fox-Landi  algorithm  truly  a one-pass  method. 

On  the  other  hand,  if  their  method  is  altered  so  that  the  item  data  are  passed  through  a 
second  time,  it  is  possible  to  eliminate  virtually  all  of  the  requirement  for  secondary  storage.  In 

the  first  pass,  only  the  running  total  cost  corresponding  to  each  0K,  K f.  [ 0 A/),  is  saved. 

At  the  end  of  this  phase,  the  "optimal"  multiplier  value,  0 *,  is  established.  The  second  phase  of 
the  algorithm  requires  a second  pass  through  the  data  base.  In  the  second  pass,  the  optimal 
stock  levels  for  each  location  are  found  for  all  items  by  resolving  problem  P8  with  0 = 0*. 

In  some  applications,  the  Fox-Landi  one-pass  method  is  clearly  infeasible;  that  is,  there 
may  not  be  enough  peripheral  storage  capacity  to  save  all  of  the  data.  If  storage  capacity  is 
available,  there  is  a tradeoff  between  the  time  and  cost  required  to  store  and  access  the  data  in 
the  secondary  memory  using  the  one-pass  method,  and  the  time  and  cost  to  recompute  the 
stock  levels  using  the  second  method.  For  realistic  Air  Force  problems,  the  two-pass  method 
appears  to  be  the  only  feasible  approach,  given  current  hardware  constraints,  if  M is  large 
enough  to  guarantee  that  a solution  can  be  found  that  closely  approximates  the  target  budget. 

The  Bisection  Method 

A third  way  to  solve  problem  PI  is  a slight  modification  of  the  Fox-Landi  algorithm  called 
the  bisection  method,  which  employs  a bisection  search  to  find  the  optimal  value  for  0.  This 
procedure  requires  initial  upper  and  lower  bounds  on  the  optimal  value  of  0.  Call  these  0 y and 
0i  , respectively.  The  bisection  method  is  as  follows: 

STEP  1:  Set  fl  - (fly  + fl,)/2. 

STEP  2:  Solve  problem  P8  with  0 — 0 for  each  item. 

^TEP  3:  If  the  total  cost  of_the  solution  obtained  in  Step  2 exceeds  C,  then  replace  0 t 
with  fl;  otherwise,  replace  fly  with  0. 

STEP  4:  If  a stopping  criterion  has  not  been  met  (such  as  a fixed  number  of  iterations  or 
an  error  tolerance),  return  to  Step  1;  otherwise,  stop. 

The  m^jor  drawback  to  the  bisection  approach  is  that  a separate  pass  through  the  item  data 
base  is  required  at  each  iteration  of  the  algorithm.  This  algorithm  performs  very  well  in  terms 
of  convergence,  and  we  have  found  that  it  almost  always  produces  solutions  that  are  within  1/2 
percent  of  the  target  budget  using  10  bisections. 

Comparison  of  Methods 

The  closeness  of  the  solutions  to  the  target  budget  generated  by  either  the  Fox-Landi 
method  or  the  bisection  algorithm  depends  on  how  broad  a range  of  multiplier  values  must  be 
searched  for  a fixed  value  of  M or  a fixed  number  of  bisections.  It  should  be  pointed  out  that 
both  of  these  methods  only  yield  an  approximation  to  the  optimal  multiplier  value  (assuming 
one  exists). 

Of  the  methods  discussed  thus  far,  it  has  been  the  experience  of  the  author,  as  well  as  of 
Fox  and  Landi  [2],  that  the  latter  two  algorithms  better  Sherbrooke's  algorithm  in  run  times  by 
an  order  of  magnitude  or  more  on  real  problems,  given  reasonable  estimates  of  upper  and  lower 
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bounds  for  the  Lagrangian  multiplier  Thus,  in  the  comparisons  we  will  report,  only  these  two 
Lagrangian  methods  will  be  discussed. 

Approximation  Methods 

Earlier  we  described  an  approximation  method  for  estimating  the  optimal  value  of  U and 
of  each  N,.  Several  options  are  open  for  implementing  this  approximation  method.  One  way  to 
implement  it  is  to  use  a two-phase  approach.  We  call  this  approach  the  First  Approximation 
Method.  The  values  of  a,  and  b,  are  computed  in  the  first  phase  of  this  method,  and  the 
optimal  value  of  0 is  estimated  using  (El).  In  the  second  phase,  we  solve  problem  P8  for  each 
item,  using  the  estimate  of  the  optimal  0.  This  approach  has  two  major  advantages  over  the 
Fox-Landi  method: 

1.  The  estimate  of  the  optimal  multiplier  can  be  obtained  without  prespecifying  a range  of 
values,  and  computation  time  to  obtain  the  estimate  does  not  depend  on  the  uncertainty  of  the 
multiplier  value. 

2.  The  computation  time  to  find  an  estimate  of  the  optimal  multiplier  is  much  smaller. 

If  the  two-pass  version  of  the  Fox-Landi  algorithm  is  used,  the  second  phase  of  that  method 
and  the  second  phase  of  the  approximation  method  are  the  same.  The  one-pass  version  of  the 
Fox-Landi  algorithm  requires  considerably  more  storage,  and  also  requires  more  computer  time 
to  determine  the  optimal  stock  levels,  than  this  approximation  method  requires. 

This  approximation  approach  also  has  advantages  over  the  bisection  method: 

1.  Only  two  passes  through  the  data  base  are  required,  as  opposed  to  seven  or  more 
required  for  the  bisection  method  in  practice. 

2.  No  stock  levels  need  to  be  saved;  in  the  bisection  method  it  is  necessary  to  save  all 
stock  levels  and  other  data  for  three  multiplier  values. 

Another  algorithm  can  be  employed  that  directly  uses  the  results  of  the  approximation 
problem,  that  is,  problem  P2.  We  call  this  approach  the  Second  Approximation  Method.  This 
algorithm  is  of  interest  in  situations  in  which  only  total  system  stock  is  computed  for  each  item, 
and  there  is  no  interest  in  computing  the  optimal  distribution  of  the  assets.  Determining  the 
optimal  allocation  of  a budget  among  items  is  of  primary  importance  when  purchasing  inventory 
or  making  budgetary  projections  for  spares  for  different  systems.  In  these  cases,  distribution 
decisions  are  usually  not  very  critical. 

The  Second  Approximation  algorithm  also  consists  of  two  phases.  In  the  first  phase  we 
estimate  the  values  of  the  a,  and  b,  parameters,  and  in  the  second  phase  we  determine  total  sys- 
tem stock  for  each  item,  using  the  algorithm  described  in  Section  III,  and  rounding  N,  to  the 
nearest  integer.  The  algorithm  requires  one  pass  through  the  item  data  base  and  one  pass 
through  an  item  file  consisting  of  a„  b„  and  c,.  The  mtqor  advantage  of  this  approach  is  that  it 
eliminates  the  stock-allocation  phase  of  both  the  Fox-Landi  method  and  the  First  Approxima- 
tion algorithm. 

V.  A COMPUTATIONAL  COMPARISON  OF  VARIOUS  ALGORITHMS 

The  Fox-Landi  algorithm,  the  bisection  algorithm,  and  the  two  approximation  methods 
have  been  coded  and  tested  on  several  sample  sets  of  data  for  the  Air  Force’s  new  F-15  fighter. 
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Since  all  of  the  tests  yielded  the  same  general  results,  we  will  discuss  only  two  of  them  in 
detail.  The  first  test  had  a 75-item  sample  and  had  3 operating  bases.  The  flying  programs 
were  very  different  at  each  base.  In  the  second  test,  125  items  were  included  in  the  sample, 
with  demands  occurring  at  5 bases;  in  this  test,  only  the  Fox-Landi  and  the  two  approximation 
methods  were  compared.  The  run  times  stated  for  both  approximation  algorithms  include  the 
time  required  to  estimate  the  values  of  a,  and  br  In  all  Fox-Landi  calculations,  a maximum  of 
128  multiplier  values  was  examined;  ten  bisections  were  used  in  all  applications  of  the  bisection 
method.  Furthermore,  in  both  test  cases  all  stock  levels  for  all  relevant  multiplier  values  were 
stored  in  main  memory.  Thus,  although  the  reported  computation  times,  which  include  com- 
pile times,  are  roughly  equal  for  all  the  algorithms,  they  are  biased  in  favor  of  the  Fox-Landi 
method  because  this  type  of  storage  would  be  impossible  for  larger  problems.  In  addition,  the 
range  of  multiplier  values  considered  in  the  tests  of  the  Fox-Landi  and  the  bisection  methods 
was  selected  after  estimating  the  optimal  multiplier  value  using  the  First  Approximation 
Method.  Thus  the  test  results  are  biased  in  favor  of  them,  since  the  range  of  multiplier  values 
was  much  smaller  than  would  normally  be  the  case. 

The  data  displayed  in  Tables  1 and  2 indicate  how  well  each  approach  approximates  a 
given  target  budget  for  the  two  test-data  sets.  Without  a doubt,  the  bisection  method  produced 
solutions  that  best  matched  the  target  budgets,  followed  in  order  by  the  Second  Approximation 
Method,  the  Fox-Landi  method,  and  the  First  Approximation  Method.  As  mentioned  before, 
the  results  are  biased  in  favor  of  both  the  Fox-Landi  and  the  bisection  methods  due  to  the  ini- 
tialization of  the  range  of  multiplier  values.  From  a practical  viewpoint,  all  approaches  worked 
acceptably  well  in  meeting  the  target  budgets.  Furthermore,  the  stock  levels  generated  by  the 
various  approaches  were  virtually  the  same  for  similar  budgets.  Consequently,  total  system 
expected  backorders,  for  all  practical  purposes,  are  indistinguishable;  that  is,  the  backorder 
versus  investment  curves  virtually  coincide  among  these  various  approaches.  Exact  comparison 
of  computed  stock  levels  and  expected  backorders  cannot  be  made  among  the  competing 
methods  since  the  allocation  of  the  available  budget  in  each  case  depends  on  the  way  each  algo- 
rithm estimates  the  Lagrangian  multiplier. 

The  area  in  which  the  methods  clearly  differ  is  in  computation  time.  The  approximation 
methods  require  substantially  less  time  than  either  the  Fox-Landi  method  or  the  more  time- 
consuming  bisection  method.  Other  experimentation  has  shown  that  the  percentage  difference 
in  computation  times  tends  to  be  even  greater  as  the  number  of  items  increases. 

Thus,  the  approximation  methods  produce  answers  that  are  as  good  as  those  produced  by 
the  Fox-Landi  method  and  the  bisection  method,  but  do  it  much  more  quickly  than  those 
methods.  The  bisection  method  does,  however,  match  target  budgets  slightly  better  than  the 
approximation  methods.  However,  the  approximation  algorithms  are  "irtually  foolproof,  which 
is  perhaps  their  greatest  advantage.  The  user  does  not  have  to  spec  the  range  of  multiplier 
values  or  the  number  of  bisections  in  advance.  This  eliminates  one  problem  associated  with 
implementing  either  the  Fox-Landi  or  the  bisection  algorithm.  In  view  of  these  observations, 
the  approximation  procedures  developed  here  appear  to  be  superior  for  use  on  real  problems. 

VI.  ESTIMATION  OF  THE  OPTIMAL  DEPOT  STOCK  LEVEL 

We  have  described  Sherbrooke's  algorithm  and  several  Lagrangian  methods  for  solving 
problem  PI,  and  have  demonstrated  that  it  is  possible  to  reduce  significantly  the  computational 
requirement  of  the  Fox-Landi  method  by  solving  an  approximation  problem  to  obtain  a good 
estimate  of  an  appropriate  value  for  the  Lagrangian  multiplier.  In  this  section  we  describe  a 
different  way  to  reduce  the  computational  requirements  of  all  the  algorithms  that  have  been 
discussed.  As  can  be  seen  by  reexamining  Sherbrooke’s  approach  (see  problem  P6)  and  the 
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Fox-Landi  algorithm  (see  problems  P8  and  P9),  the  amount  of  computation  required  to  solve 
problem  PI  using  these  methods  is  directly  proportional  to  the  number  of  depot  stock  levels 
explicitly  examined.  Consequently,  if  this  number  can  be  reduced,  then  the  total  time  required 
to  compute  an  optimal  solution  can  also  be  reduced. 

The  method  of  estimating  the  optimal  depot  stock  level  that  we  describe  in  this  section  is 
of  particular  value  when  the  expected  number  of  units  in  the  depot  resupply  system  for  an  item 
is  20  or  more.  The  approximation  algorithm  can  reduce  computation  time  for  the  algorithms 
described  in  Section  IV  by  as  much  as  90  percent  for  these  high-demand  items. 

We  have  indicated  how  the  optimal  base  stock  level  s'  can  be  calculated  given  the  depot 
stock  level  s,0  and  the  value  of  9.  In  particular,  we  have  shown  that  s'  is  optimal  if  it  is  the 
smallest  nonnegative  integer  for  which 

Z p[x|x,/7‘/(s(0)J  < 9c,. 


We  now  develop  a different  but  equivalent  way  of  characterizing  s,*.  To  simplify  notation,  let  us 
suppress  the  item  index  /.  We  will  also  assume  that  p[jr|x/7’/(s0)]  has  a Poisson  distribution. 

Define  the  convex  back-order  function  for  base  j as 

Bj(sj\  Sq)  A Z (x  - Sj)/>[x|Xy7)(so)], 

X>Sj 

for  s,  > 0 and  integer,  and  Bn  the  piecewise  linear  completion  of  Bh  as 

B,(r,  so)  if  i is  a nonnegative  integer, 

[fl,(s(;  So)  - fl,(s,  - 1;  so)][r  - (s,  - 1)) 

Bt(t\  so)  A < + B(s>  - 1;  Sq),  s,  - 1 < t < s,\ 
where  s,  is  a nonnegative  integer, 

, and  B(-l;  s o)  A °°. 


A#,(s,;  Sq)  A BfUf,  Sq)  - B,(s,  - 1;  Sq). 

when  s(  is  a nonnegative  integer,  and 

D(st\  Sq)  A {v:  Afl,(s,;  Sq)  < v < Aff,(s,  + 1;  Sq) }- 

Observe  that  D^s/.s©)  U (Afl,(s,;  sq))  is  the  set  of  subgradients  of  Bi  at  s,.  Then  an 

alternative  way  of  verifying  that  s’  is  an  optimal  base  stock  level  is  to  show  that 

— Oc  € D(s‘\  Sq). 

Next  let 


Ft* !«  *2 Jo)  A £ [fl/fi/i  Sf)  + »CS,]. 

i- 1 

By  dropping  both  the  integrality  and  nonnegativity  restrictions  on  sq,  we  obtain  the  following 
relaxation  of  problem  P8: 

(P10)  min  j9cs„  + min  [F($\,  So):  s0  fixed}). 

<o  1 v-o.i  f 
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If  s0  'S  the  optimal  solution  to  problem  P10,  then 

bF 


(E3) 

But 

Furthermore,  by  writing  Sq)  as 


dso 


+ 6c  - 0. 


bF  _ S.  6 T, 

bso  y.i  bTj  dio 


£ Kp[K  + sjxr^so)). 
k-i 


we  see  that 


K -f  11,-1 


Ml  T x Ke~K‘T^  [K'T<(s°>]  1 

97-,  "a-.  ' (*+*/-»! 


- T ATX  e~klTl(5o>  [X'T'(so>]K+'' 

1 (K+sy. 

- - X,A0(s,;  so). 

As  we  discussed  in  Section  II,  tne  function 

£o(so)  A !(»-  so)p(jr|XD) 


can  be  closely  approximated  by  an  exponential  function  of  the  form  ape  0 °,  where  a0and  b 0 
are  positive  real  numbers.  Then 

T/(so)  - r,A,  + (1  - r)Bi  + f a«e "Vo 


and 


Hi . - llzli 

ds  o X 

Upon  combining  these  observations,  we  see  that 

a so)  ~~  «oV"V#- 

Recall  that  -9c  € D(s/;  so).  Consequently  -6c  approximates  the  marginal  reduction  in 
backorders  at  base  J when  the  stock  level  at  that  base  is  s'.  After  making  this  substitution  and 
representing  this  further  approximation  of  bF/bs0  by  or/Bso,  we  see  that 


If-- 


9so  /-i 
- - Oca  oboe 

Substituting  this  approximation  into  (E3)  we  obtain  the  following  estimates  of  the  optimal 
depot  stock  level: 


-•  v V • 


INVENTORY  SYSTEMS  FOR  RECOVERABLE  ITEMS 


393 


(E4) 


1 

ao^o 


Recall  that  the  value  of  C0  is  derived  based  on  an  exponential  approximation  of  Bn(sJ. 
As  the  average  number  of  units  in  the  depot  repair  cycle  increases,  that  is,  as  \D  increases,  the 
quality  of  this  exponential  approximation  improves  in  the  region  in  which  the  optimal  depot 
stock  level  should  be  located.  Consequently,  the  approximation  should  be  most  accurate  in 
these  cases.  But  the  problems  for  which  the  search  for  the  optimal  depot  stock  level  is  most 
time-consuming  for  the  algorithms  described  in  Section  IV  correspond  to  the  items  having  a 
large  number  of  units  in  depot  repair.  Therefore,  the  proposed  approximation  method  will  be 
most  appropriate  for  the  items  requiring  the  greatest  amount  of  computational  effort. 

The  approach  we  have  described  for  estimating  the  optimal  depot  stock  level  has  been 
coded  and  tested  using  a sample  of  40  F-15  aircraft  items.  The  test  consisted  of  two  sets  of 
runs.  In  the  first  set,  monthly  flying  was  divided  among  3 bases;  in  the  second  set  the  same 
monthly  flying  program  was  divided  among  5 bases.  The  total  budget  distributed  among  the  40 
items  ranged  from  X34  million  to  $65  million  in  the  first  set  of  runs,  and  from  $34  million  to 
$88  million  in  the  second  set.  Table  3 contains  the  data  indicating  both  the  optimal  and  the 
estimated  depot  stock  levels  for  each  item  in  both  runs. 

As  shown  in  the  table,  there  is  usually  no  single  optimal  depot  stock  level  for  an  item. 
Rather,  the  optimal  value  depends  on  the  amount  of  total  item  system  stock  available  for  distri- 
bution among  the  depot  and  bases.  The  estimate  of  optimal  depot  stock  is  quite  close  to  the 
optimal  value  in  all  cases.  Furthermore,  the  increase  in  expected  system  backorders  using  the 
estimated  depot  stock  levels  rather  than  the  optimal  levels  is  generally  small.  For  most  items, 
the  increase  is  substantially  less  than  0.1  back  orders. 

The  results  of  the  tests  indicate  that  it  is  possible  to  estimate  closely  the  optimal  depot 
stock  level  using  (E4).  Additionally,  incorporating  this  method  for  estimating  the  optimal 
depot  stock  into  the  algorithms  described  in  Section  IV  will  considerably  reduce  the  search 
required  to  find  the  optimal  depot  stock  level,  and  will  therefore  markedly  reduce  the  computa- 
tional time  needed  to  solve  problem  PI  using  these  algorithms. 
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AN  INVENTORY  DEPLETION  PROBLEM  WITH  RANDOM  AND 
AGE-DEPENDENT  LIFETIMES 
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ABSTRACT 

The  following  problem  is  studied  The  units  of  an  inventory  are  used  one 
by  one  until  all  have  failed  Their  lifetimes  decrease  with  their  ages,  when  they 
are  taken  out  of  the  inventory  An  item  of  age  a is  supposed  to  have  a lifetime 
Y exp(-o),  where  Y is  a random  variable  which  does  not  depend  on  a.  It  is 
shown  that  in  order  to  maximize  the  total  lifetime  the  items  should  be  taken 
according  to  the  LIFO  principle  This  is  shown  for  a certain  class  of  distribu- 
tions of  Y This  class  includes  the  exponential  and  the  Pareto  distributions. 


1.  INTRODUCTION 

Consider  a stockpile  consisting  of  n items,  which  are  characterized  by  their  age.  When  an 
item  of  age  a is  taken  out  it  will  last  for  the  time  T(a).  When  it  fails  it  is  immediately  replaced 
by  a new  one  until  there  are  no  more  items  left.  If,  for  instance,  there  are  only  two  items  in 
the  stockpile,  with  ’es  ax  and  o2,  the  total  lifetime  will  be  either  7’(a1,  a2 ) — 
T(ax)  + r[o2  + T(a ,)j  or  T(a2,  ax)  - T(.a2)  + T[ax  + r(a2)J,  depending  on  the  order  in 
which  the  items  are  taken  out.  Lieberman  [4]  asked  in  what  order  the  items  should  be  taken  in 
order  to  make  the  total  field  life  as  long  as  possible.  He  and  later  Brown  and  Ross  [1]  gave 
almost  complete  answers  when  the  lifetime  is  a fixed  function  of  the  age. 

In  this  paper  we  shall  study  a special  case  of  random  lifetimes.  We  suppose  that  they  are 
distributed  as 

(1.1)  Y exp(-ca), 

where  Y is  a random  variable  and  c is  a constant.  We  will  prove  that  the  LIFO  principle  is 
optimal  for  a class  of  distributions  of  the  random  variable  Y.  This  class  includes  among  others 
the  exponential  distribution.  LIFO  is  short  for  "Last  In  First  Out,"  i.e.  the  youngest  item  in 
stock  shall  always  be  the  one  that  is  taken  out.  Ross  (Ref.  [S],  p.  179)  stated  this  as  an  open 
question.  He  conjectured  that  the  LIFO  principle  would  be  optimal.  Brown  and  Solomon  [2] 
studied  random  lifetimes. 

In  the  next  section  we  introduce  the  notations  and  do  some  preliminary  work.  In  the 
third  section  we  discuss  the  paper  of  Brown  and  Solomon  and  its  relation  to  our  work.  Our 
main  result.  Theorem  4.1,  is  given  and  proved  in  the  following  section.  The  last  section  con- 
tains two  examples,  in  which  Theorem  4.1  holds. 
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2.  PRELIMINARIES 

When  the  lifetimes  are  fixed  it  is  clear  what  is  meant  by  maximal  total  field  life.  When 
they  are  random  this  is  not  so  obvious.  Ross  [5]  considered  expected  total  field  life.  We  will, 
however,  use  the  same  criterion  as  Brown  and  Solomon  12],  stochastic  monotonicity.  A ran- 
dom variable  X is  stochastically  larger  (or  smaller)  than  Y if  its  distribution  function  lies  to  the 

d 

right  of  and  below  (or  to  the  left  of  and  above)  that  of  K We  will  write  this  as  X > Y 

</ 

(or  X < Y).  (If  they  have  the  same  distribution  we  use  an  equality  sign  ) It  is  easy  to  see  that 
X is  stochastically  larger  than  Y if  and  only  if  E[h(X))  ^ E[h(  K) ] for  all  increasing  functions 
/»,  such  that  the  expected  values  exist.  If  we  take  h to  be  the  identity,  this  is  the  criterion  of 
Ross. 

We  will  also  use  the  monotone  likelihood  ratio  (m.l.r.).  The  random  variable  X has 
increasing  (or  decreasing)  likelihood  ratio  with  respect  to  Y if  /, ( r )/y  > ( r ) increases  (or 
decreases)  with  r.  Here  fx  and  fy  denote  the  densities  of  X and  Y.  It  is  easy  to  see  that  the 
monotone  likelihood  ratio  implies  stochastic  monotonicity.  If  every  pair  of  random  variables  in 
a family  has  a monotone  likelihood  ratio,  we  will  say  that  the  family  has  the  m.l.r.  property 
An  equivalent  definition  of  m.l.r.  is  fx(x)  fy(y)  - fyiy)  fyix)  > 0 whenever  x ^ v. 

We  will  also  use  failure  rates  [3].  Suppose  that  the  random  variable  .Yhas  the  distribution 
function  Fand  the  density  f.  Its  failure  rate  is 

ttU)  _ /(/)/[  1 - F(r)]  - lim  P(X  < i + ‘i  | X > t)/h. 

h~ 0 

An  absolutely  continuous  distribution  is  completely  defined  by  its  failure  rate: 

F(x)  - 1 - expl-  nit)  di]. 

In  the  following  we  shall  only  study  positive  random  variables.  If  a distribution  is  defined  by  a 
failure  rate  such  that  Fix ) does  not  converge  to  one,  we  say  that  the  random  variable  takes  the 
value  infinity  with  probability  exp]-  / fi(t)  dt ].  Some  results  for  failure  rates,  which  are 
needed  later  on,  are  given  below. 

LEMMA  2.1:  Let  X and  Kbe  two  independent  random  variables  with  failure  rates  ft , and 
ft  Y.  Then  the  failure  rate  ft  of  minlX  Y)  is 

ft(l)  — ftx(t)  + M y(t)-. 


PROOF:  The  failure  rate  ft(t)  equals 

lim  -7  P(X  < t+h  or  f < t+h  \ X > t and  Y > t) 
h—X)  h 

- lim  ~ [PiX  t+h  I X > t)  + P(Y  ^ t+h  | Y > t) 

/i— o h 

- PiX  < t+h  | X > t)  ■ P(Y  < t+h  | Y > r)] 

— ft  x(t)  + ft  y(t)  ~ 0. 

LEMMA  2.2:  If  X and  Y are  two  positive  random  variables  such  that  X has  increasing 
likelihood  ratio  with  respect  to  Y then  ftx(x)  < ft  y(x)  for  all  positive  x. 
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PROOF:  The  proof  is  by  contradiction.  Suppose  that  for  some  value  x„, 

/*(*„)  _ fy(x0) 


i;U<)  dt  > f’fyU)  dt 

0 o 

Using  the  assumption  of  m.I.r.,  we  have  the  following  contradiction, 


- IX  y(x„). 


/*(*) 

/ 

fy(x) 

fyix) 

S ~fxU)  dt 

/ 

frit)  dt 

JTfyO)  dt 

*o 

dx 


s: 


fy(> r) 


X Tv (0  dt 


dx  - 1. 


If  instead  of  two  random  variables  we  consider  a multiplicative  family  with  the  m.I.r.  pro- 
perty we  get  the  following  corollary. 

COROLLARY  2.1:  If  X is  a random  variable  and  if  the  family  aA'has  the  m.I.r.  property 
then  xn  x(,x)  increases  with  x. 

PROOF:  The  failure  rate  of  aX  is  nx(x/a)/a.  By  the  previous  lemma  this  is  decreasing 
in  a and  the  result  follows. 

LEMMA  2.3:  Let  Ybe  a random  variable.  Let  h(  ) be  a convex  function  such  that  /r(0) 
- 0 and  h (x ) - xis  increasing.  If  xnx(x)  increases  with  xthen  nx(x)  > nh(X)(x). 

PROpF:  Since  h(x)  is  convex  it  must  be  differentiable  almost  sverywhere  and 
h 0)  - Jo  h'(x)  dx.  Define  h'(t)  to  be  right  continuous  in  the  points  where  the  derivative 
does  not  exist.  The  function  h'(x)  so  defined  is  increasing  in  x because  h(x)  is  convex.  It  is 
now  easy  to  see  that 

h(t)/t  - f h' (x)  dx/t  < t ■ h'(t)/t  - h'(t). 

•'0 

The  conditions  of  the  lemma  give  that  /»(/)  ^ rand  hence 

t/ix(t)  < h(t)  /xxlh(t)]. 

Using  these  two  formulas  we  have 

Hh(xM(t)]  - iix(t)/h'(t)  < nx[h(t) ] hU)/[t  /r'(f)l 
^ !ix[h(t)]. 

With  x - hit)  the  result  follows. 

3.  RELATION  TO  PREVIOUS  WORK 

Brown  and  Solomon  [2]  studied  random  lifetimes,  but  they  did  not  consider  exponential 

d d d 

deterioration  for  other  than  bounded  variables.  Let  X\  ^ X7  > ...  > X„  be  independent  ran- 
dom variables  and  d a positive  function.  An  item  / will  last  for  the  time  X,d(t)  if  it  is  kept  in 
stock  for  the  time  t before  it  is  taken  out. 
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Brown  and  Solomon  showed  that  if  d is  increasing  and  convex  the  items  should  be  taken 
in  the  given  order.  If  on  the  other  hand  d is  increasing  and  concave  they  should  be  taken  in 
the  opposite  order.  They  4lso  showed  that  if  d is  convex  and  PiXx  > X2)  - 1,  the  items 
should  be  taken  out  with  increasing  indexes.  We  will  here  give  a simple  corollary  of  this  result. 
The  corollary  says  that  if  all  the  lifetimes  are  bounded  by  M and  if  dit)  does  not  decrease  fas- 
ter than  MM.  then  the  items  should  be  taken  with  increasing  indexes. 


THEOREM  3.1:  Suppose  that  Xx,  X2 X„  are  independent  random  variables  such 

d d d 

that  -T)  ^ X2  ^ ...  > X„  and  X2  < M with  probability  one.  If  d is  positive  decreasing  and 
convex  and  if  |rf'(0)|  < 1/A/  then  the  items  should  be  taken  in  the  given  order. 


PROOF:  If  X\  > M.  the  result  follows  immediately  from  the  results  of  Brown  and  Solo- 
mon, so  let  us  assume  that  X\  < M.  Let  T0  - A/ and  define  git)  by 


di  0) 

1 4-  Md'iO ) 


+ d'(0)t  if  t 


Md(0) 
\+ Md'iO) 


d 


Mdj  0) 

1 + Md'iO) 


if 


> A*d(0) 

"■  1 + Md'iO) 


The  result  of  Brown  and  Solomon  can  now  be  applied  to  X0,  ...  , X„  and  git).  Thus 
X0,  X,,  ... , Xn  should  in  this  situation  be  taken  in  this  order.  But  when  X0  fails  we  have 
exactly  the  situation  described  by  our  theorem  and  the  result  follows. 

Brown  and  Solomon  also  proved  that  if  there  are  only  two  items  with  m.l.r.  in  the  stock- 
pile, the  larger  one  should  be  taken  first  when  d is  positive  and  convex.  We  will  here  prove  a 
weaker  result  using  a proof  that  illustrates  one  of  the  tricks  used  in  proving  Theorem  4.1.  First 
we  state  a simple  lemma  without  proof. 

LEMMA  3.1:  The  function  Xi  + x2exp(— ex,)  - x2  — x,  exp(— cx2)  is  positive,  increas- 
ing, and  convex  as  a function  of  x,  > x2. 

Denote  the  initial  ages  of  the  items  in  stock  by  a,  < a2  and  the  total  field  life  by 
Tiah  a2),  if  the  item  with  initial  age  a,  is  taken  first,  and  by  Tia2,  a,)  if  the  items  are  taken 
in  the  opposite  order.  We  consider  from  now  on  the  situation  described  by  (1.1),  i.e.,  with 
exponential  deterioration. 

LEMMA  3.2:  Let  the  family  aY.  a t R+,  have  the  m.l.r.  property.  Then 
Ha,,  a2)  ^ Tia2,  ax). 

d 

PROOF:  Let  Xx  and  X2  be  independent  random  variables  such  that  Xx  - Y exp(-ca,) 
and  X2  - Y exp(-ca2).  With  this  definition  we  have 

Xx  + X2  exp(— c2f|)  - Tiax.  a2) 

(3.1)  ^ 

X2  + Xt  exp i-cX2)  - Tia2,  a() 

If  the  densities  of  Xx  and  X2  are  denoted  by  fx  and  /2,  respectively,  then  their  joint  density  can 
be  written 

/i(*i)  /2Uj) 

“ f/K-X'i)  Aix2)  /(x,  < x2)  + /|(x2)  /2(x,)  /(x,  > x2)l 
+ \ix  fl(x2)  - /|(X2)  /2(X|))  /(x,  > x2). 


(3.2) 
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where  / is  the  indicator  function.  The  first  term  is  symmetric  in  X|  and  x2.  The  second  term  is 
positive  when  x,  > x2  and  vanishes  elsewhere.  This  follows  from  the  assumption  of  m.l.r. 

In  order  to  obtain  the  distributions  of  T(a ,,  a2)  and  T(a2,  a |),  we  integrate  the  two 
terms  over  the  sets  x,  + x2  exp(-cx,)  < t and  x2  + x,  exp(-cx2)  ^ /,  respectively.  From  the 
symmetry  it  follows  that  the  integrals  of  the  first  term  are  equal.  It  follows  from  Lemma  3.1 
that  the  integral  of  the  second  term  is  smaller  over  the  first  set.  Lemma  3.2  now  follows  from 
the  definition  of  stochastic  monotonicity. 

REMARK  3.1:  This  method  of  dividing  the  density  functions  can  also  be  used  when  n — 

3.  In  that  case,  however,  the  density  has  to  be  divided  into  six  rather  complicated  terms,  all  of 
which  require  different  proofs.  We  will  not  try  to  do  so  here  since,  as  far  as  we  know,  there  is 
no  possibility  of  generalizing  this  method  to  a general  n. 

4.  A GENERAL  NUMBER  OF  ITEMS 


Let  the  items  in  stock  have  ages  a,  < a2  < a}  < ...<  a„  and  let 

T(a, (1),  a,(2) a,(n))  denote  the  total  field  life  when  the  item  with  age  al(l)  is  taken  first, 

that  with  initial  age  a,(2)  is  taken  next,  and  so  on.  If  we  assume  that  the  LIFO  principle  is 
optimal  when  the  stockpile  consists  of  p items,  and  if  n = p + 1 we  know  that  the  p last  items 
will  be  taken  according  to  the  LIFO  principle.  The  youngest  om  'ill  thus  be  one  of  the  first 
two  items  taken.  If  we  could  show  that 

T(at,  ak,  a2,  ...  , a*_ j,  ak+i,  ...  , ap+ ,) 

(4.1)  t 

^ T(ak,  a |.  a2 i.  «*+i.  •••.  flp+i) 

the  result  would  follow,  for  by  optimality  of  the  LIFO  principle  when  n — p,  the  first  quantity 
is  always  stochastically  smaller  than  T(a  a2,  ....  ap+i). 

Formula  (4.1)  can  be  rewritten  as 

r(a,,  ak ) + r[a2+  r(alr  ak) ak_\  + Tfa,,  ak), 

ak+i  + T(a j,  ak) a„+1  + 7(a,.  ak )) 

d 

> T(ak,  a,)  + f(a2  + T(ak,  a,) a*_,  + T(ak,  a,), 

(4.2)  ak+l  + T(ak,  a,),  ...  , ap+ \ + T(ak.  a,)]. 

The  problem  can  thus  be  viewed  as  a p-unit  problem,  where  the  first  item  has  a different  distri- 
bution (here.  T(a ,,  ak)  or  T(ak,  a,),  respectively).  Lemma  4.1  treats  this  problem.  We 
repeat  that  p.  is  the  failure  rate  of  the  variable  Kin  the  definition  (1.1)  of  lifetime  distributions. 


LEMMA  4.1:  Suppose  that  the  LIFO  principle  is  optimal  when  n — p.  Let  £/,  and  U2  be 
random  variables  with  failure  rates  and  p2  and  let  the  following  conditions  hold: 


(4.3) 


x explc(y  + a,)l 


> /x[(jc  +y)  exp(cy)]  exp(ca,), 


(4.4)  Mi(*)  ^ M2(->f)>  and 

(4.5)  Mi(jf)  < mU  exp(cai))  exp(ca,). 
The  following  stochastic  inequality  holds: 

f,  - (/,  + Ha,  + (/, an.x  + (/,) 
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> T2  - U2  + T(a,  + U2 o,_,  + U2). 


PROOF:  Let  us  suppose  that  we  have  the  situation  described  as  7",,  i.e.  p units  such  that 
the  first  unit  has  the  lifetime  U\  and  the  other  units  have  lifetime  distributions  corresponding 
to  their  initial  ages  a,,  a2 an^]. 

Construct  Z,  with  the  failure  rate  (i)  = p.2(t)  — H] (/),  which  by  (4.4)  is  positive.  Now 

</ 

by  Lemma  2.1  we  have  f/2  — min(Z),  U\).  If  we  replace  the  first  unit  at  time  min(Z,,  U |), 
even  though  it  may  be  functioning,  the  total  field  life  will  be  distributed  as  T2.  The  general 
idea  of  the  proof  is  to  change  minfZ),  (/,)  into  U]  in  a number  of  steps  so  that  each  step  will 
yield  a stochastically  larger  field  life. 


If  Z | ^ U ,,  we  are  finished.  If  Z , < Uh  we  define  the  variable  U2  as 

(4.6)  t/3  — — Z,. 

The  failure  rate  of  U2  is  p.\(t  + Z|).  If  Z\  < (/,  the  remaining  total  field  life  after  Z,  is  distri- 
buted as 

(4.7)  r(fl)  + Z, a„.  , + Z,) 

or  as 

(4.8)  U}  + T(.at  + Z,  + U3 + Z , + UJ 

depending  on  whether  we  replace  the  first  item  at  Z,  or  not. 


construct  a new  random  variable  Z2  depending  on  Z,  by  its  failure  rate 


(4.9) 


A,(/) 


I explc (a  | + Z ,)] 


exp[c(o,  + Z ,)]  - mi('  + Z\), 


which  by  conditions  (4.3)  and  (4.5)  is  positive.  This  definition  leads  by  Lemma  2.1  to 

(4.10)  min(f/3,  Z2)  - Yg(al  + Z,). 

If  we  replace  neither  at  Zx  as  in  (4.7)  nor  at  U\-Z\  + U2  as  in  (4.8)  but  at 
Z,  + min((/3,  Z2),  the  remaining  total  field  life  at  Z,  will  be  distributed  as 

T(a  \ + Z \,  j + Z|,  a2  + Z\,  . . . , ap-\  + Z\). 

By  the  induction  hypothesis  this  is  stochastically  larger  than 

T(a \ + Z\,  <ip_|  + Z|,  a i + Z\)  ^ T(a \ + Z\,  ap_|  + Z j) . 

In  other  words,  we  have  shown  that  the  total  field  life  is  stochastically  longer  if  the  first  unit  has 
the  lifetime 

(4.11)  min(Zi,  U\)  + min(Z2,  U2)  /(Z,  < U\)  - min(Z,  + Z2,  l/,). 
than  if  the  first  unit  has  the  lifetime 


(4.12) 


min(Z),  U\)  - U2. 


The  first  step  in  the  change  of  min (Z,.  Ut)  into  U \ is  now  completed.  The  next  step  will 
proceed  in  exactly  the  same  way  with  Z , replaced  by  Z,  4-  Z2.  If  Z,  + Z2  > Uh  we  are 
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finished;  otherwise,  we  define  t/4—t/|  — Z|  — Z2  [see  (4.6)].  It  has  the  failure  rate 
M i(t  + Zj  + Z2).  Construct  Z,  depending  on  Zt  and  on  Z2  with  the  failure  rate  [see  (4.9)] 


+ Z|  + Z2)  - M|(/ + z,  + Z2). 


d 

Now  [see  (4.10)]  we  have  min(f/4,  Z3)  - Y g(a,  + Z,  + Z2).  We  thus  find  that  the  total 
field  life  is  stochastically  larger  if  the  first  item  has  the  lifetime  [see  (4.11)  and  (4.12)] 
min(Z,  + Z2  + Z3,  (/,)  than  if  it  has  the  lifetime  l/2. 


The  second  step  is  completed.  By  repeating  this  argument  unitl  Zx  + Z2  + . . . ^ (/,  we 
find  that  it  would  be  better  to  let  the  first  item  remain  in  the  system  until  it  fails,  i.e.  until  the 
time  Uy 

That  Z\  + Z2+  ...  eventually  will  reach  U\  follows  from  the  fact  that  A,(/)  is  bounded 
in  every  finite  interval 


X ,(/)  < sup  *i[ t exp(cx)]  exp(cx)  for  /'  > 1. 

0 

The  distribution  of  (/2  is  thus  changed  into  that  of  (/,  in  such  a way  that  the  total  field  life 
increases  stochastically  in  each  step. 

THEOREM  4.1:  If  the  following  conditions  hold: 

(4.13)  n (x)  decreases  in  x, 

(4.14)  Y ■ a,  aeR+  has  the  m.l.r.  property,  and 

(4.15)  x fi'(x)/i iz(x)  decreases  in  x, 
the  LIFO  principle  is  optimal. 

r 

PROOF:  It  is  trivial  that  the  theorem  holds  for  n — 1.  We  have  in  fact,  by  Lemma  3.2, 
that  it  holds  for  n — 2,  but  we  do  not  need  that  fact.  We  will  now  prove  that  if  it  holds  for 
n — p,  then  it  also  holds  for  n — p + 1.  When  this  is  done  the  result  follows  from  the  axiom 
of  induction.  Unfortunately  it  is  not  possible  to  use  Lemma  4.1  directly  on  (4.2).  We  will 
therefore  first  change  the  situation  into  one  where  the  lemma  is  applicable  and  then  check  the 
conditions.  Exactly  as  in  the  proof  of  Lemma  3.2  we  find  that  it  is  sufficient  to  consider  the 
situation 

X{  + Xk  exp(-c*,)  - T(a{,  ak) 

Xk  + Xx  exp (~cXk)  - T(ak,  a,) 

where  the  distribution  of  Xt  and  Xk  is  given  by  AT[/i(x|)  fk(.xk)  — f\(xk)  /*(x,)]  /(xi  > x*). 
In  order  to  simplify  notation  we  introduce  the  following  positive  variables: 

Z - Xk  + Xk  exp (-cXk), 

Y,  - X,  + Xk(-cX,)  - Z,  and 

Y2  - (X,  - Xk)  exp (~cXk). 


402 


D. THORBURN 


Formula  (4.2)  is  now  reduced  to 

Y , + T(a2  + Z + Y «*_,  + Z + Y„  ak+,  + Z + Y, + Z + Y,) 

> Y2+  T(a2  + Z + Y2 fl*_,  + Z + Y2.  ak+l  + Z + Y2 ap+l  + Z + Y ,). 

This  formula  will  be  shown  to  hold  for  any  given  fixed  Z by  use  of  Lemma  4.1.  Observe  that 
a,  in  the  lemma  corresponds  to  al+i  + Z or  al+2  + Zhere,  depending  on  whether  / < k — 1 or 
not.  The  failure  rate  of  Y\  and  Y2,  given  Xk,  are  called  mi  and  n2. 


Condition  (4.3)  is  easily  shown  to  follow  from  (4.13)  and  (4.14)  and  Corollary  2.1: 
exp|c(o2  + Z + y)|  expjc(a2  + Z + y)j| 

> expjc(fl2  + Z + y)J  /uJ(jf  + y ) exp|c(a2  + Z + y) jj 

> exp|c(o2  + Z)]  /x|(jc  + y)  exp[c(a2  + Z) jj. 


Condition  (4.4)  is  more  complicated.  If  xn2(x)  were  increasing.  Lemma  3.1  would  say 
that  K,  - K2  is  positive,  increasing,  and  convex  as  a function  of  K2,  and  Lemma  2.3  that 
condition  (4.4)  held  for  such  functions.  We  will  thus  try  to  show  that 


( x - Xk) 


f\(x)  fk(xk)  -/w  AU) 


(4.16) 


X /,(f)  h(Xk)  - /,(**)  fkU)  dt 

AU)  f,(xk) 

I — 

#t|(x  - Xk)  exp(ca,)J  x exp(co,) 


fi(x)  fK(Xk) 


1 - 


fji\x  exp(cfl))l  exrTca!)  fk(x)  fx(Xk) 
u[x  exp(cfl*)l  exp(c<j*)  A(x)  fk(Xk) 


is  increasing. 


By  Corollary  2.1  the  first  factor  is  increasing.  The  m.l.r.  property  gives  that  the  second 
term  of  the  numerator  decreases  from  one  as  x increases  from  Xk.  That 

^t(x  exp(c  a,)]  exp(c  a,) 
li[x  exp(c  a*)]  exp(c  ak ) 

is  less  than  one  and  increasing  in  x follows  from  Lemma  2.2  and  the  fact  that  the  derivative  of 
its  logarithm 

ix'[x  exp(co,)l  #i'lx  exp(cot)l 

exp  cfl,  exp(Cfl|)]  exp  fi[x  exp(co*)]  ' 

is  positive  [by  (4.15)]. 


That  the  second  factor  of  (4.16)  is  also  increasing  can  now  be  seen  by  writing  it  as 
(4.18)  (l  - y)/[  1 - h(y)  y\. 

where  y equals  the  second  term  of  the  numerator  of  (4.16)  and  h(y)  equals  (4.17).  Formula 
(4.18)  is  easily  seen  to  be  increasing  in  x noting  that  its  derivative  with  respect  to  y is  negative. 
We  have  now  shown  that  (4.16)  is  increasing  in  x > Xk. 
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Condition  (4.5)  remains  to  be  proved.  If  x > Xk  the  failure  rate  of  X}  equals 

- Xk)  exp(-c**)|  exp(-c^) 

_ f\W  fk(xk)  - /|(xt)  fk(x) 

~fx  /l(')  dt  fk(xk)  - /,(x*)  fkU)  dt 
< /,(*)/  £~  /.(/)  dt 
- /u[x  exp(ca,)]  exp(ca,). 

The  first  expression  follows  from  the  definition  of  K2.  The  inequality  is  a consequence  of  the 
m.l.r.  property  and  Lemma  2.2.  A redefinition  of  xand  (4.13)  and  (4.14)  gives 

M ,(x ) < ti-^x) 

< Aijx  exp|c(a,  + 2f*)J  + Xk  exp(ca,)|  exp|c(a,  + JTA)J 


Condition  (4.5)  follows  if  we  remember  that  ai  in  Lemma  4.1  corresponds  to  a2  + Xk  in  this 
proof. 

5.  APPLICATIONS 

It  is  easy  to  check  that  the  conditions  of  Theorem  4.1  hold  for  the  exponential  distribu- 
tion. The  failure  rate  is  in  that  case  a constant.  Condition  (4.13)  that  /x(x)  is  decreasing  and 
condition  (4.14)  that  x /i'(x)/fi(x)  is  decreasing  are  both  trivial.  Condition  (4.15)  that  the 
exponential  distribution  has  the  m.l.r.  property  is  well  known. 

Another  class  of  distributions  for  which  the  conditions  hold  is  the  Pareto  family  starting  at 

zero: 

(5.1)  fix)  a (x  + a)-*,  a >0,  k > 1. 

The  failure  rate  of  this  distribution  equals 

(5.2)  M(x)  - (k  - l)/(x  + a). 

It  is  easily  seen  that  (5.2)  is  decreasing  in  x.  The  m.l.r.  property  holds  since 
fy(x)/fay(x)  - (x  - l)_*/[(l/a)  (x/a  - l)]-* 

- [1  + (a  - l)/(x  - a)l'*a/_* 
is  increasing  in  x if  a > 1.  Condition  (4.15)  is  easy  to  check: 

xn'(x)/n(x)  - x[—(k  - 1 )/(x  + aH/K k - l)/(x  + a)] 

- -1  + a/(x  + a). 
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ABSTRACT 

This  paper  considers  Ihe  problem  of  computing  IHX";  X > i)  when  V is  a 
normal  variale  having  Ihe  properly  lhal  Ihe  mean  is  substantially  larger  than 
Ihe  standard  deviation  An  approximation  is  developed  which  is  determined 
from  the  mean,  standard  deviation,  and  the  cumulative  standard  normal  distri- 
bution Computations  comparing  Ihe  approximate  moments  with  Ihe  actual  are 
reported  for  various  values  of  ihe  relevant  parameters  These  results  are  ap- 
plied to  the  problem  of  computing  the  expected  number  of  shortages  in  a lead- 
time  for  a single  product  which  exhibits  continuous  exponential  decay 


1.  THE  APPROXIMATION 


The  purpose  of  this  paper  is  to  consider  an  approximation  for  X>  t)  for  t >0 

when  A'  is  a normal  variate  with  the  property  that  /x»  <r , which  is  common  in  physical 
processes  in  which  a normal  distribution  is  used  to  describe  a nonnegative  phenomenon.  Since 
the  probability  of  obtaining  a negative  observation  is  .(—  /z/<r)  (where  d>  is  the  cumulative 
normal  distribution)  it  must  be  true  that  n»  <r  in  order  that  this  probability  be  neligible.  The 
application  of  our  results  to  a problem  associated  with  a product  which  exhibits  continuous 
exponential  decay  will  also  be  considered.  The  random  variable  X will  correspond  to  the  lead- 
time  demand  which  is  a nonnegative  random  variable  generally  assumed  to  be  normal. 


We  have  that 

(1) 

where 


£(*-*  X > t)  - f”  x~”f(x  | fi,  (r)  dx 


f(x  | m.  <r)  - exp 


— Or  - fi)2/2  it2 


For  any  n ^ 1,  this  computation  would  require  using  numerical  integration  methods 
The  approximation  developed  here  is  based  on  approximating  the  normal  density  by  a gamma 
density,  simplifying,  and  then  reapproximating  the  resulting  gamma  density  by  the  normal. 

Since  n »<r,  f(x)  can  be  approximated  by 
•Research  Supported  by  the  National  Science  Foundation  under  grant  F.NG  75-04990 
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l 


- 


! 


where 


gix  | a.p)  = x“-'  e~** 

a — ifi/cr)2  and  P = /x/cr2. 


The  justification  for  this  is  as  follows.  It  is  well  known  that  g (x  | a,  /3)  is  the  probability 
density  of  the  sum  of  k independent  and  identically  distributed  random  variables  with  denstiy 
gix  | a/k,  p)  (see  DeGroot  [1],  p.  237).  It  follows  from  the  central  limit  theorem  that  for  a 
large,  gix  | a,  p)  can  be  approximated  by  a normal  density.  The  mean  and  variance  of  a 
Gamma  random  variable  with  parameters  a and  p are  ft  = a/p  and  a-2  = a/p2.  Solving  for  a 
and  p in  terms  of  n and  <r2  gives  the  results  above.  Note  that  the  original  assumption  that 
n »<r  implies  that  a will  be  large  and  gix  \ a,  p)  will  give  a good  approximation  to 
fix  | n,  a). 

Hence 

Ei) T";  X >t)  = f ™ x-"g(x  | a,  p)  dx 

_ f . P"  x"~'~ne~p'dx 
Via) 

(2)  - 7 — tt-L r J -pf r xa~n~ie~fix  dx. 

(a  — 1 ) ...  (a-n)  T ia—n ) 


The  integrand  of  (2)  is  simply  the  gamma  density  gix  \ a - n,  p).  Since  this 

gives  a »0  which  would  imply  that  a — n»  0 for  moderate  values  of  n (that  is, 
n»  aVn).  Hence,  we  may  than  reapproximate  j?(x  | a— n,  p)  by  a normal  density  with 

mean  u - and  variance  <r2  - “ --  . It  follows  that 
P p2 


EiX  ",  X > r)  = 


JL 


(«-l) ... 

ia—n) 

P" 

(a-1)  ... 

ia—n) 

Hi. 

1 

z^yf,  I A-  ») 


dx 


<D 


1ZJL 

& 


(3) 


where  4>  (x)  - 1 - <t>(x). 


h ifi2-i  tr2) 

/ — I 


(T  Vu'  — n<rl 


The  value  of  n should  be  small  enough  so  that  /x  > aVn.  Otherwise  the  argument  of  «I> 
in  (3)  will  not  be  defined.  This  is  only  a mild  restriction  since  (1)  will  be  essentially  zero  for 
large  n.  The  quality  of  the  approximation  will  clearly  depend  upon  whether  or  not  a-n  is 
sufficiently  large  to  apply  the  central  limit  theorem  and  reapproximate  #(x  | a,  p)  with 
fix  | fi.,  &). 


In  order  to  compare  the  effectiveness  of  (3)  as  an  approximation  to  (1)  we  have  estimated 
(1)  by  numerical  integration.  Table  1 presents  a summary  of  some  of  the  computational  results 
for  various  values  of  n,  <r,  l,  and  n.  Note  that  for  fi/a  “ 10,  the  agreement  is  extremely 
close,  while  the  quality  of  the  approximation  deteriorates  as  n/o-  decreases.  It  should  be  noted 
that  for  larger  values  of  n,  (2)  may  be  used  directly  if  tables  of  the  cumulative  gamma  distribu- 
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TABLE  1 — Comparison  of  Exact  and  Approximate  Calculations 


M 

(T 

/ 

n 

Exact  value  (1) 

Approximation  (3) 

10 

1 

1 

i 

0.10103158 

0.10101009 

5 

1.175  x 10~s 

1.165  x 10"5 

10 

1 

0.04664270 

0.04648207 

5 

3.57  x 10”6 

3.55  x 10~6 

10 

2 

1 

1 

0.10461799 

0.10416607 

5 

2.931  x 105 

1.915  x 10~5 

10 

1 

0.04375230 

0.04382718 

5 

2.75  x 10  6 

2.56  x 10  6 

10 

3 

1 

1 

0.1 1252050 

0.10963436 

5 

6.0250  x 10  4 

5.102  x 10  5 

10 

l 

0.04133040 

0.04156939 

5 

2.23  x 10  6 

1.13  x KT6 

10 

4 

1 

1 

0.12173662 

0.11646283 

5 

261  x 10  1 

3.31  x 10  4 

10 

1 

0 03925619 

0.0397417 

5 

1 87  x 10  h 

4.89  x 10  8 

lion  are  available  However,  in  most  eases  where  n is  relatively  large,  it  is  generally  true  that 
it  < \4*  (with  equality  v..  !0  ng  when  X is  approximately  Poisson),  so  that  the  approximation 
should  give  excellent  results 


2.  DECAYING  INVENTORIES 


When  inventory  levels  decline  by  a fixed  fraction  each  period  due  to  spoilage  or  loss  in 
value  (exclusive  of  demand),  the  inventory  exhibits  continuous  exponential  decay.  The  term 
exponential  decay  arises  in  the  following  manner.  Suppose  l(t)  represents  the  inventory  level 
for  a continuous  review  system  at  time  t.  Ignoring  demand,  the  decay  assumption  may  be 
expressed  as  /(/  + s)/l(t)  - y'  where  y — the  fraction  of  stock  decaying  per  unit  time 
(0  < y < 1)  and  t.s  > 0,  This  is  equivalent  to 

(4)  / (/  + s)  — exp  (-Os)  I (/) 


where  0 - - 1 n (y).  The  constant  0 is  ordinarily  referred  to  as  the  decay  rate  and  represents 
the  instantaneous  rate  of  decrease  of  the  relative  inventory  level  lit  + s)/l  (t)  at  s - 0 (that 


ls’  T 

ds 


d /(/  + s) 


lit) 


,-o“  “ »)■ 


Decaying  inventories  arise  in  various  situations.  One  example  is  radioactive  materials, 
such  as  nuclear  medicines  (see  Refs.  (2]  and  (61)  or  U02  nuclear  fuel  for  a fission  reactor. 
Another  possible  application  for  an  inventory  model  with  decay  is  In  the  area  of  cash  flow 
management.  Continuous  discounting  of  future  returns  and  continuous  exponential  decay 
correspond  to  precisely  the  same  physical  process.  Alternatively,  a decay  model  might  be  use- 
ful for  providing  an  approximation  to  the  more  complex  problem  of  managing  a fixed-life  per- 
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ishable  commodity  (see  Nahmias  [5],  for  example,  for  an  analysis  of  the  periodic  review  ver- 
sion of  this  problem). 

There  is  an  alternative  interpretation  of  the  decay  model  which  explains  more  clearly  how 
it  relates  to  a perishable  inventory  problem.  Suppose  / (0)  - N units  are  on  hand  initially  at 
time  f - 0,  and  also  assume  that  the  lifetime  of  each  of  the  units  is  a random  variable  having 
the  negative  exponential  distribution.  That  is,  if  T\,  T2  ....  7\  are  the  respective  lifetimes  of 
each  of  then  N items,  the  P\T,  > s)  = exp  (— 0.v)  for  s > 0 and  1 ^ / < N.  It  follows  that 
the  number  of  units  on  hand  at  any  time  v > 0 is  a random  variable  having  the  binomial  distri- 
bution with  parameters  N and  p = exp  (-0s),  so  that  the  expected  inventory  level  at  time  s, 
say  /(s),  is  given  by  Np  or  /(0)  ■ exp  (-0s)  which  agrees  with  (4)  for  / = 0.  It  is  important  to 
recognize  that  the  exponential  decay  process  is  a purely  deterministic  process  which  arises  as 
the  expected  value  of  a process  with  random  lifetimes.  For  large  A,  the  law  of  large  numbers 
and  the  concept  of  ensemble  averages  provides  a justifiction  for  using  the  deterministic 
exponential  decay  model  for  a problem  of  this  type. 

A common  inventory  policy  for  continuous  review  inventory  systems  is  to  trigger  an  order 
when  the  level  of  on-hand  inventory  reaches  r,  the  reorder  point  (see  Hadley  and  Whitin  [4], 
Chapter  4).  An  important  part  of  determining  the  proper  value  of  r is  to  compute  the  expected 
number  of  units  which  go  short  during  the  leadtime,  say  r.  Under  the  common  assumption 
that  leadtime  demand  is  a normal  random  variable,  we  will  show  how  the  results  of  the  previ- 
ous section  can  be  used  to  give  a very  close  approximation  for  the  expected  shortage  in  lead- 
time  for  a decaying  inventory. 

Ghare  and  Schrader  (3]  have  analyzed  the  extension  of  the  simple  EOQ  model  to  allow 

for  continuous  exponential  decay.  They  show  that  if  the  depand  rate  at  time  u is  given  by  the 

function  D(u),  0 < u < f,  then  /(/)  = e ■ ( / (0)  — f D(u)  eH“  du).  When  D(u)  — A, 

•'0 

independent  of  «,  this  reduces  to 

(5)  /(r)  - (1(0)  + \/0)  - X/0. 

Suppose  that  X — demand  in  a leadtime  r.  Then  the  demand  rate  during  leadtime  is  a 
random  variable  X (X)  — X/t  which  now  must  be  treated  as  an  explicit  function  of  the 
leadtime  demand  X.  We  will  assume  that  items  leave  stock  at  a constant  rate  of  X (T)  during 
leadtime.  If  T * is  the  amount  of  time  that  elapses  from  the  time  an  order  is  placed  until  inven- 
tory level  hits  zero,  then  for  any  realization  X of  the  leadtime  demand,  the  inventory  level  /(f) 
will  decrease  according  to  (5)  with  X — X(x),  on  the  interval  between  the  time  the  order  is 
placed  until  min  (T*  ,r).  The  case  T*  < t is  pictured  in  Fig  1. 

It  follows  from  (5)  that  for  a given  value  of  x,  T*  solves 

0 - e~*T'(r  + x(x)/0)  - X(x)/0 
or 

(6)  f*  - (1/0)  • In  (1  + r0/X(x)). 


In  order  for  a shortage  condition  to  occur  it  is  necessary  that  T*  < t.  Let  w(r)  be  the 
minimum  number  of  units  demanded  in  a leadtime  to  ensure  that  a shortage  condition  will 
occur.  Then  w(r)  may  be  obtained  from  (6)  with  T*  — r and  X(x)  — w(r)/r,  which  gives 
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Figure  I — A possible  realization  of  the  inventory  level,  /(r),  in  a leadtime. 


or 

(7)  w(r)  - r#  r/(e"T  - 1). 

Hence,  if  X < wir),  then  T*  < r and  no  shortage  occurs,  while  if  X > wir),  then  T*  < r and 
a shortage  will  occur. 

Suppose  that  yix)  - numbers  of  units  which  decay  in  a leadtime  when  x is  the  leadtime 
demand.  When  T*  < r,  the  inventory  level  drops  from  r to  0 in  a time  T *,  which  includes  the 
losses  due  to  both  demand  and  decay.  The  loss  due  to  demand  is  exactly  A^x)  • T*.  It  follows 
that  the  loss  due  to  decay,  y(x),  is  given  by  yix)  - r - A(x)  T*,  which  from  (6)  becomes 

(8)  y(x)  - r - In  (1  + r9/\(x)). 

It  now  follows  that  for  all  x ^ w(r),  the  number  of  units  which  go  short  in  a leadtime, 
say  Six),  is 

S(x)  - x + .y(x)  - r, 

which  becomes,  after  we  substitute  A(x)  - x/t,  and  use  (8), 

(9)  Six)  - x - ix/0r)  • In  (1  + rOr/x). 


Since  total  demand  in  leadtime  is  assumed  to  be  normal  with  mean  n and  variance  <rJ, 
obtain  the  expected  number  of  units  short  in  the  leadtime,  say  Pir),  from  (9)  as 


we 


Pir)  - EiSiX).  X > 


r 

J wir ) 


w(r)) 

1 


(10) 


x - ix/0r)  ■ In  (1  + r 9 t/x)\  fix\n,  cr)  dx, 
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where  w(r)  is  given  by  (7).  Note  that  for  x = w(r)  the  integrand  will  vanish.  We  may  write 
P(r)  - P\  ( r ) - P2(r)  where 


P'(r)  “ /,(,)  x a)  dx 

and 


P2(r)  “ f (x/0t)  In  (1  + r 9t/x)  f(x\ n.a)  dx. 

•'w(r) 


It  is  well  known  (see,  for  example,  Hadley  and  Whitin  [4],  p.  446)  that  since  fix  \ fi,  a) 
is  a normal  density. 


P\(r)  “ a <f> 


w(r) 


where  <t>  is  the  standard  normal  density  and  <t>  is  the  complementary  cumulative  standard  nor- 
mal distribution  function.  Note  that  we  also  used  the  identity 


fix  | ix.  it)  - — d> 

(T 


Unfortunately,  no  such  direct  result  is  available  for  determining  P2(r),  so  that  exact  com- 
putation of  P2(r)  requires  numerical  methods.  However,  if  we  use  a Taylor  series  expansion 
for  the  In  (1  + r 9 t/x),  the  results  of  the  previous  section  can  be  applied.  The  Taylor  series 
expansion  for  In  (1  + /)  around  any  point  /„  can  be  shown  to  be 

n 

(13)  In  (1  + /)  - In  (1  + /„)  + I (-D"'1  ~ T~^~ 

" l+'o 

where  t — r 9 t/x. 


We  have  experimented  with  various  values  of  t0  and  found  that  the  best  results  were 
obtained  at  t0  -*  r 9 t/h.  In  this  case  (13)  becomes 


******  X 


j 


INVERSE  MOMENTS  FOR  DECAYING  INVENTORIES 


411 


It  is  of  interest  to  examine  the  conditions  under  which  this  series  will  converge.  Suppose 
that  n - 3 a ^ x ^ n + 3a- . Then 


< 1 for  (*s  < f 
6 


rOr 

"|i_±r 

*-i 

l + Hl. 

U A 

< 

P 

whenever  |x  - mI  < 3o\  which  guarantees  convergence  of  the  series  for  this  range  of  values  of 
x.  Note  that  this  agrees  with  our  former  requirement  in  Section  1 that  a should  be  small  in 
comparison  with  /u. 

Using  (14)  and  truncating  after  N terms,  we  now  obtain  the  following  approximate 
expression  for  P2ir ): 


(15)  P2ir)  ~ 


In 

1 + ^2- 

P 

Or 

1 N 
I 


1 N i 

°T  n- 1 n 


fw(r)  x fix  \p.tr)  dx 

ot) 

! 


r0  r 


1 + 


rflr 

P 


w(r) 


if" 

/-  0 


/ 

X 

fix  |m.  <t)  dx 


_1_ 

Or 


In 


1 + 


rOr 


n 1 

+ i (-I)"-1  • - ■ 


rOr 

n 

-l 

X + I0j_ 
P 

M 

P,ir) 


1 N 

+ i(-i )•-• 

*T  A 1 1 + 


i n i 

+ TT  IC-D-'i 

97  n-2  n 


rOr 


rOr 

P 

rOr 


-1 


n-\ 


$ mJjl  -p 

<T 


1 + 


rOr 


iw  t 


\n- 1 


r /<*iy-  *> 

3»(r)  v'-I 


The  final  equality  results  from  separating  out  the  terms  corresponding  to  /—  0 and  /—I  and 
using  the  definition  of  Ptir),  an  explicit  expression  of  which  is  given  in  (11).  The  final  term 
can  now  be  approximated  by  using  (3),  with  the  result  that 


C /<*!/*.  a)  dx- * 

w(r)  J./-I  /-l 


/-I 


II  in 2 - ktr1) 

A-l 


(16) 


$ [/*»>(/■)  - ■ n2  4-  (/  - 1)  (T1 
a-y/fi1  - ii  — 1)  crJ 
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Combining  (11),  (15),  and  (16)  gives  the  following  computing  formula  for  P(r): 


(17) 


P(r) 


1 "^ln 


1 + 


rAr 


+ JL  y 1 'flr 

At  “ n M + rAr 


IT  4> 

w(r)  — n 

+ n 4> 

w(r)  - u. 

(T 

cr 

- JL.  * *.<'>  ~ M y 

At  tr  " 


tAt 


cr 

r0r 


/?  — l 


fx  + rOr 


M* 


w,-l 


where 


A* 


— Vjt2  — /c<r2, 
P 


4>  (x)  - -1=  e"*2/2,  <l>  (x)  - 4,(1 ) dt. 


Although  (17)  may  appear  to  be  fairly  complicated,  it  is  completely  determined  from  the 
knowledge  of  only  the  mean  and  variance  of  leadtime  demand  and  the  standard  normal  density 
and  distribution  functions.  Computations  were  performed  to  compare  the  effectiveness  of  the 
approximation  for  P(r ) for  various  combinations  of  m,  o',  t,  and  A.  The  exact  calculation  of 
P(r)  given  in  (10)  was  accomplished  by  numerical  integration  and  the  approximation  was 
obtained  from  (17).  Table  2 summarizes  the  results  for  n — 10,  a — 2,  r — 5,  and  /V  = 6 for 
A - 0.1,  0.3,  0.7,  and  1.0.  Note  that  the  agreement  is  quite  close  throughout  and  is  exact  to 
two  decimals  for  all  10  < r < 130  when  A - 1.  Also,  the  values  of  w(r)  can  be  seen  to 
change  substantially  with  A.  We  also  tested  the  approximation  for  larger  values  of  /z  and  found 
that  it  gave  excellent  results  as  long  as  fi/i r was  relatively  large  (say  6 or  more).  It  is  worth 
pointing  out  that  although  (17)  is  fairly'complicated,  it  can  be  determined  far  more  quickly  for 
moderate  values  of  A than  can  (10)  by  numerical  integration.  This  can  be  extremely  important 
when  the  expected  number  of  shortages  incurred  during  leadtime  must  be  computed  for  a range 
of  values  of  r. 
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TABLE  2 — A Comparison  Between  Exact  and  Approximate  Expression 
for  Expected  Shortage  for  p = li),  <r  = 2,  N =■=  6,  r =5. 
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ABSTRACT 

An  exact  method  for  solving  all-integer  linear-programming  problems  is 
presented  Dynamic-programming  methodology  is  used  to  search  efficiently 
candidate  hyperplanes  for  the  optimal  feasible  integer  solution.  The  explosive 
storage  requirements  for  high-dimensional  dynamic  programming  are  avoided 
by  the  development  of  an  analytic  representation  of  the  optimal  allocation  at 
each  stage  Computational  results  for  problems  of  small  to  moderate  size  are 
also  presented 


INTRODUCTION 

The  problem  to  be  considered  in  this  paper  is  the  all-integer  linear-programming  problem, 
which  is  as  follows: 


max  z — c'x, 

Ax  < b,  and 

(1)  x > 0,  integer. 

In  (1),  c and  x are  n-component  vectors  and  c has  integral  nonnegative  components.  A is  an 
m x n matrix  and  b is  an  m-component  vector.  Furthermore,  we  assume  that  z is  bounded  and 
that  there  exists  at  least  one  feasible  integer  point  in  the  convex  set  5 - {x|/lx  < b,  x > 0). 

In  an  earlier  paper  by  one  of  the  authors  [1],  a precursor  of  the  formulation  of  the 
method  proposed  in  this  paper  was  made.  However,  it  was  not  efficient  from  the  point  of  view 
of  computer  storage.  A basic  reformulation  of  the  method  is  made  in  this  paper  which  changes 
the  previous  algorithm  very  significantly.  In  the  present  paper,  the  basic  theory  to  make  the 
algorithm  computationally  efficient  is  presented.  The  new  algorithm  is  presented  along  with 
some  computational  results.  It  should  be  emphasized  that  this  algorithm  is  exact  and  suffers 
from  no  numerical  problems  of  convergence. 

The  assumption  that  the  components  of  c are  nonnegative  is  equivalent  to  the  statement 
that  each  of  the  separable  terms  of  the  objective  function  is  nondecreasing.  This  is  required  by 
the  dynamic  programming  approach  we  shall  use.  This  is  not  a limitation  in  generality,  since  if 
any  of  the  ct,  J — 2,  ...  , n,  are  negative,  a simple  transformation  may  be  made  to  convert 

the  negative  coefficients.  This  will  be  discussed  in  a subsequent  section. 
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It  will  be  noted  that  if  any  attempt  were  made  to  solve  the  problem  given  by  (1)  by 
dynamic  programming  it  would  rapidly  run  into  the  "curse  of  dimensionality,"  for  m > 3 or  4 
and  the  storage  requirements  on  a computer  would  be  not  only  prohibitive  but  nonexistent. 
The  method  proposed  here  avoids  this  explosive  increase  of  memory  requirement  when  the 
dimensionality  is  high.  We  will,  however,  exploit  the  use  of  dynamic-programming  methodol- 
ogy, as  will  be  seen  subsequently. 

2.  GENERAL  DESCRIPTION  OF  THE  ALGORITHM 

The  general  idea  of  the  proposed  algorithm  is  to  search  candidate  hyperplanes  for  lattice 
points.  This  proceeds  as  follows.  If  we  remove  the  integer  requirement  from  (1),  the  relaxed 
problem  can  be  solved  as  a linear-programming  problem.  Suppose  the  optimal  value  of  the 
objective  function  for  this  relaxed  problem  is  z°.  In  addition,  let  the  optimal  value  of  the  objec- 
tive function  for  (1)  be  designated  z*.  It  is  clear  that  z*  < z°.  The  basic  idea  behind  the 
hyperplane  search  algorithm  we  propose  is  to  start  at  the  linear-programming  solution  and 
search  the  hyperplane  c'x  = [z  1 (where  [al  indicates  the  greatest  integer  less  than  or  equal  to 
a)  to  see  whether  or  not  it  contains  any  feasible  lattice  points.  If  it  does,  we  are  done.  If  it 
does  not,  we  move  the  hyperplane  in  a direction  parallel  to  itself  and  then  search  the  hyper- 
plane c'x  - [z  1-1.  Since  c was  assumed  to  have  integral  components,  c'x  must  be  an  integer 
if  x is  to  have  all  integral  components.  If  the  hyperplane  c'x  - [z  1 - 1 contains  at  least  one 
feasible  lattice  point,  we  are  done.  If  it  does  not,  we  continue  the  process.  This  procedure  is 
clearly  finite.  Since  it  was  assumed  that  S was  nonempty  and  contained  at  least  one  lattice 
point,  we  must  eventually  find  it.  Let  us  now  describe  this  algorithm  or  class  of  algorithms 
more  precisely. 

We  summarize  the  notation  we  will  use: 

z*  — optimal  value  of  objective  function  in  (1), 

z°  — optimal  solution  to  linear-programming  problem  derived  from  (1), 

z*  - z°  - k.  k - 0.  1,  2 

S — [x\Ax  < b,  x ^ 0), 

2y  - lower  bounds  on  xy,  i.e.,  Xj  >1,  J - 1,  2 n,  and 

Uf  — upper  bounds  on  x/t  i.e.,  xy  < ■ 1,  2 n. 

HYPERPLANE  SEARCH  ALGORITHM 

1.  Solve  the  linear  programming  problem 

max  z - c'x 
Ax  < b,  and 

(2)  x > 0. 

If  xp,  the  optimal  solution  to  (2),  satisfies  the  requirement  of  being  all  integer,  we  are  done. 
Otherwise,  proceed  to  step  2. 

2.  Determine  lower  bounds  lt  and  upper  bounds  ut  for  each  variable  x,.  We  then  have 

1/  ^ Xt  < U/, 

(3)  x i integer,  j - 1,  2 n. 
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3.  Find  all  combinations  of  x„  j - 1,  2 n , which  satisfy 

! c'x  = Zk 

(4)  l ^ x,  < y'  = l,2 n. 

4.  If  no  integer  valued  vector  x*  can  be  found,  increase  A:  by  1,  i.e.,  decrease  zk  by  1 and 
return  to  step  3 (or  in  some  cases  step  2 — see  discussion  below).  If  at  least  one  xA  is  all 
integer,  go  to  step  5. 

5.  If  at  least  one  xk£S,  we  are  done.  If  for  all  x\  xk6S,  decrease  zk  by  1 and  return  to 
step  3 (or  in  some  cases  step  2 — see  discussion  below). 

Let  us  now  consider  each  step  of  the  above  algorithm  in  greater  detail.  First,  we  may 
note  that  since  the  set  5 is  nonempty,  is  bounded,  and  contains  at  least  one  integer  point,  the 
finiteness  of  the  algorithm  is  guaranteed.  How  efficient  such  an  algorithm  can  be  depends  very 
strongly  on  how  step  3 is  carried  out.  Let  us  consider  each  step  in  turn. 

Step  1 requires  little  comment.  Any  simplex  code  can  be  used  to  solve  the  linear- 
programming  problem  given  by  (2). 

Step  2 should  be  carried  out  in  the  most  convenient  fashion.  Lower  bounds  of  zero  on 
the  Xj  can  always  be  used.  If  upper  bounds  can  be  easily  determined  from  the  physical  intepre- 
tation  of  the  variables,  they  should  be  used.  Linear-programming  could  also  be  used,  if  neces- 
sary, to  determine  both  the  lower  and  upper  bounds  on  the  x,  by  solving  the  problems 

min  xt,  max  x/: 

Ax  <6,  Ax  < b. 

c'x  = zk,  c'x  = zk. 

(5)  OO.  x ->0. 

These  problems  could  be  solved  once  (initially)  to  get  bounds,  or  periodically,  as  steps  4 and  5 
of  the  algorithm  indicate.  In  any  case,  lower  and  upper  bounds  can  be  found. 

Steps  4 are  5 and  self-evident  and  require  no  extensive  comment  except  to  note  that  this 
is  the  only  place  the  structural  constraints  of  (1)  enter  the  problem,  with  the  possible  exception 
of  the  determination  of  bounds.  As  we  shall  see,  the  determination  of  the  solution  x*  in  step  3 
, does  not  explicitly  depend  upon  the  constraints  Ax  ^ b. 

3.  HYPERPLANE  SEARCH  BY  DYNAMIC  PROGRAMMING 

We  now  consider  how  we  may  use  a dynamic  programming  formulation  and  method  of 
solution  to  deal  with  step  3 of  the  hyperplane  search  algorithm.  This  amounts  to  finding  all 
combinations  of  xt  which  satisfy  (4).  We  may  formulate  this  problem  as 

' ^ max  z - c'x 

c'x  - zk, 

c,  > 0,  j - 1,2 n, 

0 < JL,  < Xj  < U/,  J » 1,  2,  ....  n.  and 

x f integer,  j - 1,  2 n. 


i 
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The  fact  that  we  already  know  the  maximum  value  of  z for  this  subproblem,  viz,  zk,  in  no  way 
invalidates  (6)  as  a meaningful  problem,  since  what  we  are  seeking  is  whether  or  not  there 
exists  a set  of  values  x satisfying  the  constraints  of  (6).  It  will  be  noted,  as  mentioned  previ- 
ously, that  we  need  to  assume  that  all  c,  > 0.  Let  us  demonstrate  that  this  does  not  result  in 
any  loss  of  generality. 

LEMMA  1:  An  integer  programming  problem  of  the  form  given  by  (6),  except  that  not 
all  of  the  objective  function  coefficients  are  positive,  may  be  transformed  into  an  equivalent 
problem  all  of  whose  coefficients  are  positive. 

PROOF:  The  proof  is  by  construction.  Suppose  we  have  the  problem 

max  z — c'x, 
c'x  - zk, 

(7)  l < x < u, 

where  c is  an  integer  vector,  but  not  all  c,  > 0.  If  c,  > 0,  let  x,  = x,  and  c,  = c/t  and  if  c,  < 
0,  left  x,  - u,  - x i and  c,  - — cy.  Further,  let  P = {v'|c,  > 0}  and  N - [/'|c/  < 0).  We  may 
then  rewrite  the  objective  function  of  (7)  as 

max  z - £ CjXj  + ]£  CjXj 

JtP  JtN 

- Lc,x,+  TcM-Xj) 

/•P  i'N 

(8)  - c'x  + £ CjUj. 

Hence,  instead  of  solving  (7)  directly,  we  may  solve  the  equivalent  problem 


max  z = c'x. 


(9) 

c'x  = Zk  + £ CjU)  = zk,  i < X < 

/€/V 

where 

i-i, 

if  c,  > 0, 

(10) 

a,  - w, 

and 

K 

1 

O 

A 

o 

(11) 

M,  - U, 

-i 

and  now  the  equivalent  problem  (9)  has  all  c , > 0. 

r • * 

The  problem  given  in  (6)  can  be  solved  by  the  use  of  dynamic  programming.  Since  the 
objective  function  and  the  single  structural  constraint  are  separable  and  nondecreasing  ( ct  > 0) 
functions,  the  sufficient  conditions  for  a solution  by  dynamic-programming  are  satisfied  (see 
Ref.  121).  By  applying  the  principle  of  optimality  the  dynamic-programming  solution  to  (6)  is 
easily  obtained.  The  optimal  return  functions  £,(•)  are  given  by  the  following  recursion  rela- 
tions: 

02)  tfiU)  - max  c,x,  - lx  , X - c,r,  / - 0,  1 u,. 

f.  ( , *l“®l  I olherwiie. 
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whers 

(13) 

(14) 


#SU) 


max 

/,<*,<  «,(X) 


— . A — cy,  i—O,  1,  . 

c i 

undefined,  otherwise. 


s - 2,  3,  .. 

[c,**  + ' c5jcs)),  x _ 0 , 


. , n, 

. . A, 


(15) 


8, (A) 


(16) 


As 


L ciui- 


/-i 


The  usual  dynamic-programming  approach  would  be  to  calculate 
g,(k)  and  xf‘(A),  \ - 0,  1 As, 

for  s = 1,  2 n — 1,  where  x,"(A)  is  the  value  of  x,  which  produced  gs(k)  for  each  value 

of  A.  Finally,  we  would  calculate  gn(zk ) and  x„’(z*),  assuming  a solution  exists.  We  would  then 
subtract  c„x’„  from  zk  and  then  find,  corresponding  to  A - z*  - c„x'„,  in  the  tabulation  of 
(A),  the  value  of  x„_,  ( zk  — c„x’„)  which  gave  rise  to  g„-\(zk  — c„x„’).  This  backwards  pro- 
cess would  yield,  successively,  x'„,  x„’_[ x,‘. 


The  principle  difficulty  with  this  approach  is  the  storage  requirement,  which,  while  it  is 
orders  of  magnitude  less  than  that  for  the  simple-minded  approach  of  using  a state  variable  for 
each  constraint  of  (1),  still  is  quite  considerable.  For  each  variable  a vector  xs‘(A)  must  be 
stored.  Furthermore,  there  are  often  many  alternate  optimal  values  of  x,‘(A)  which  maximize 
cjx!  + £/r-|(A  — c,xs)  and  they  must  all  be  stored.  Hence  xs*(A)  is  actually  a matrix,  say  of 


average  dimension  ~ 


u, 

y x A, 


Hence  the  total  amount  of  storage  required  is  approximately 


£ ~ - — • For  example,  if  all  c,  were  5,  all  w,  10,  and  n ~ 100,  we  would  require  about 

i-i  l 


1,260,000  words  of  computer  storage.  There  are  ways  to  minimize  this,  but  nevertheless,  with 
increasing  n,  the  storage  problem  becomes  significant. 


In  the  following  section,  a set  of  equations  will  be  derived  by  the  use  of  simple  combina- 
torial theory,  which  will  give  explicit  formulae  for  x,’(A)  for  any  A,  and  hence  the  need  for  a 
complete  tabulation  of  x,*(A)  will  be  eliminated.  Indeed  g„(k)  need  never  be  explicitly  calcu- 
lated. The  reduction  in  storage  is  drastic  and  renders  the  hyperplane  method  just  described  of 
practical  use.  The  entire  calculation  process  will  be  reduced  to  calculating  x’(zk), 

*/i-i  (z*  - Cnx„),  . . . , x,‘(z*  - £ c,x,')  directly. 

i-2 

4.  DERIVATION  OF  EQUATIONS  FOR  OPTIMAL  SOLUTION 


The  derivations  that  follow  are  concerned  with  deriving  a set  of  expressions  that  yield 
x,'(A)  by  applying  the  recursion  relations  (12)  to  (16)  to  the  problem 
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max  z = £ c, x,.  (C|  = 1) 

/-I 

n 

Z c>x,  - z*. 

1-1 

l,  < x,  < W,,  7 -1.  2 n, 

(17)  c(  integer,  j - 1,  2 

It  will  be  noted  that  we  have  assumed  that  C|  - 1.  This  is  not  absolutely  necessary  but  it  does 
result  in  a drastic  simplification  of  the  expressions  for  x,*(\).  There  is  no  loss  in  generality  in 
doing  so,  as  the  following  lemma  shows. 

LEMMA  2:  Given  the  problem 

max  z — c'x, 
c'x  = zk, 
l $ X < u, 

(18)  c > I,  integer, 

a problem  satisfying  the  conditions  of  (17)  can  be  derived  which  contains  the  solution  to  (18) 
as  a subset. 

PROOF:  The  proof  is  by  construction.  If  we  add  a variable  x„  — 0 to  the  vector  x then 
the  following  problem  has  a solution  which  contains  the  solutions  to  (18)  as  a subset. 

n 

max  z - x„  + £ c jX i , 

1 

n 

X„  + £ c,Xj  - zk. 

/-I 

l < X ^ 14. 

X„  - 0, 

(19)  c integer. 

By  adding  the  constraint  x„  - 0 to  Ax  ^ b of  the  original  integer  linear  programming,  we  force 
x„  to  be  zero  and  hence  have  a problem  which  has  as  a subset  the  solution  to  (18). 

In  what  follows  then,  we  always  assume  that  c I — 1 . We  shall  present  the  equations  for 
optimal  allocations  x,‘(A),  s - 1,  2,  ....  n and  proofs  of  their  validity  in  the  next  set  of  lem- 
mas. Without  any  loss  of  generality,  we  shall  assume  that'/,  - 0,  s - 1.  2 n.  That  this  is 

so  is  obvious,  since  if  l < x < u and  if  y — x —l.  o < u — l — u.  Therefore,  the  prob- 
lem for  which  we  are  developing  a set  of  expressions  for  x*(A)  is: 

n 

max  z - £ CfX,  (f!  - 1), 

/-i 

n 

L <•,*/  - **• 

i-i 

0 < X/  < u j,  J — 1,  2 n, 

ct  integer,  J - 1,  2 n. 


(20) 
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LEMMA  3:  xJ'(X)  - X,  0 < X < w,. 


I 

* i 


PROOF:  The  dynamic-programming  solution,  which  results  from  the  application  of  the 
principle  of  optimality  to  the  first  stage,  is: 


(21) 


#i(X)  — max  x{  — X,  0 < X ^ integer. 


This  must  be  the  case  because,  in  the  backward  allocation  process,  the  first  stage  is  reached  last. 
If  Cj  — 1 and  X is  left  to  allocate,  then  clearly  x^X)  - X. 

LEMMA  4:  x,*(X)  - 0,  X < c,  - 1 As_„  s - 2,  3 n. 

PROOF:  The  dynamic  programming  recursion  relations  for  2 < s < n are  as  follows 
(22)  g,(X)  - max  [csx}  + £,_|(X  - c,x,)l. 

Let  X - c,x,  » f > 0.  If  X - csxs  < 0,  gs_,(^)  does  not  exist. 

CASE  1 X - c,  - 1: 

We  have 


and 


If 


and 


c,  - 1 - c,x,  - t,  cs  > 1, 
cs(l  - xs)  - 1 - f,  c,  > 1. 
xs  - 0,  f - cs  - 1,  gs_,(£)  - { - c,  - 1 < Aj_|, 


x,*(X)  - 0. 

If  Xj  > 1,  f > 0.  Hence  does  not  exist.  x/(X)  — 0 is  the  only  solution. 
CASE  2,  X < c,  - 1: 


Let 


then 


and 


If 


and 


X - c,  - t,  1 < t < c5, 

X - cs,  - c,  - i - cs,  “ € > 0, 
c,(l  - x,)  - / - f. 

x,  - 0,  f - cs  - i > 0.  g,_,(f)  - c,  - i < A,_j, 


xs*(X)  - 0. 

If  x,  > 1,  f < 0.  Hence  g,_!  does  not  exist.  x,*(X)  - 0 is  the  only  solution. 
We  have  therefore  proved  that: 

x/(X)  - 0,  X < c,  - 1 < A,_|,  s - 2,  3 n 


; 
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I 
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LEMMA  5:  x,'(X)  - 0,  1.  2 8,(X),  c,  < X < A,_,, 


where 


8S(X)  - min 


for  s - 2,  3,  ....  n. 


(23) 


PROOF:  The  dynamic-programming  recursion  relations  for  2 < s < n are: 
g,(X)  - max  [c,x,  + gs_i(X  - c,x,)]. 


Since  X ^ c,  and  8,(X)  - min 


. 


, then  X - c,x,  > 0.  Furthermore,  X < A,_i  and 


g,_,()  and  x,_|(’)  have  been  defined  for  all  values  0 < £ < A^.  Hence,  it  is  clear  that 
#j(X)  - X and  hence  a value  may  be  assigned  for  each  x„  0 < x,  ^ 8,(X),  corresponding  to 
each  term  in  brackets  in  (23).  Therefore, 

c,  < X < A j_i, 

xs‘M  “0,  1,  2 MX).  s_2i3 


x/(X) 

where 


LEMMA  6: 

X — (Aj_|  + 1) 


c. 


+ 1. 


X - (A,_,  + 1) 


+ 2,  ....  vs_i,  vs;  As_i  < X < A, 


and 


v,  - w,  - t*.  X > [c,(u,  - f*)J, 

t * - max(c,(u,  — t)  — X]  <0,  / — 0,  1,  2 s - 2,  3 n. 

I 

PROOF:  The  dynamic  programming  recursion  relations  are 


g,(X)  - max  [c,x,  + g,_i(X  - c,x,)l,  A,_i  < X < A5. 


(24) 

We  may  rewrite  (24)  as 

2c,  + £,_|(X  - 2c,),  .... 

*,(X)  - max(0  + *,_i(X),  c,  + g,_,(X  - c,),  8(x)  Cj  + ff_,(X  _ 6j(X)  Cj)]- 

Since  X > A,_,  and  g,_,(X)  for  X > A._,  is  undefined,  it  is  clear  that  x,’(X)  * 0.  Therefore, 
the  minimum  value  of  x,’(X)  > 0.  To  determine  the  minimum  value  we  note  that 

X — c,x,  < A,_i  + 1; 


Therefore, 

(25) 


x,  > 


X - (A,_i  + 1) 


However,  x,  must  be  an  Integer.  Hence  we  know  that 

f X — (A,_|  + 1) 


(26) 


x,  > 


and 

(27) 
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xs  ^ 0. 

The  above  facts  can  be  combined  by  means  of  the  following.  Since  x’  ^0,  A > 
A j_|  +•  1 + c,.  If  we  substitute  X — A,_i  + 1 + cv  into  (26)  we  have 

Aj-i  + l+  cs  - (As_|  + 1) 


However,  a value  of  1 under  this  condition  is  not  possible  since 


X,_i(As_l  + 1 + c,  - c, ) = gs_,(Av_,  + 1). 
and  ff,-i(As_l  + 1)  is  undefined.  More  generally,  if  X - A,_,  + q , then 

jf,(X)  - max(0  + £5_,(At_,  + q),  ct^_,(A(_,  + q - ct) 

6,(X)  c,  + XT, _ i ( A v _ ! + q - 8V(A)  <?,) 


will  contain  terms  for  which  gs_)(A)  is  not  defined.  This  will  occur  precisely  for  those  terms  for 
which  q > cv  This  leads  to  the  minimum  value  for  x,*as 


(28) 


X — (Aj_|  + 1) 

c, 


+ 1. 


Larger  values  of  x,'will  obviously  be  permitted,  since  X - csxs  will  decrease  as  x,  increases,  and 
hence  values  of  x,_i(  ) will  exist  for  these  arguments.  However,  there  is  an  upper  bound  on 
x,*.  This  will  be  called  v,  and  is  derived  as  follows.  If  X ^ csus , then  clearly  the  largest  value 
of  x,  is  t/j  since 

X,(c,w,)  - max[0  + g,_,(c!«!).  ct  + g,_i(cs«,  - c,) csu,  + g,_,(0)] 

It  is  clear  that  no  entry  beyond  the  last  is  possible  and  this  corresponds  to  x,  - u,.  However,  if 
X < c,w„  we  wish  to  find  the  largest  value  of  xs  compatible  with  that  value  of  X.  We  recall 
that  when  X > c,u„  the  largest  value  of  x,  - u ,.  Let  us  now  suppose  that 

(29)  X ^ c,v,  - cs(m,  - i*) 

In  order  to  make  v,  as  large  as  possible  in  (29)  it  is  clear  that 

l*  - maxlc,(w,  - f)  - A]  < 0.  I - 0,  1,  2 

I 

i.e.,  I * is  the  minimum  value  of  t such  that  X > csv,  - c,(w,  - /*).  Then  v,  - u,  - t *.  It  is 
seen  that  when  X ^ c,u,,  t*  - 0. 


We  have  not  considered  the  possibility  that  vs  may  be  less  than 
The  significance  of  this  is  treated  in  Lemmas  7 and  8. 


X — (Aj_|  + 1) 


+ 1. 


Lemmas  3,  4,  5,  6 together  constitute  the  following  theorem  which  gives  the  formulae  to 
be  used  in  the  backwards  recursion  for  a solution  by  dynamic  programming  of  the  problem 
stated  in  (20).  The  optimal  values  of  x,(X)  for  any  X and  for  all  s can  be  calculated  from  these 
equations. 

THEOREM  1:  The  optimal  returns  x,*(X)  for  any  X and  all  j which  constitute  the  solution 
to  (20)  are  included  in  the  following: 
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(30) 

(31) 

(32) 

(33) 


r 


x,’(X)  -X,  0 < X < U|,  X integer, 

0,  X < c,  - 1 < A,.],  s - 2,  3 n, 

0,  1,2 8S(X),  cs  < X < A,_1(  s - 2,  3 n. 


X — (As_i  + 1) 


+ 1, 


X — (Aj_i  + 1) 


+ 2 vs_,,  vs, 


x,*(X)  - < 


s - 2,  3 n. 


where  v,  - u,  - t *,  X > [cs(us  - r*)], 

/*  - max[cs(«s  — f)  - X]  < 0,  t - 0,  1,  2,  ... 

I 

and 

(34)  ^undefined  if  X > As,  s - 2,  3,  ...  , n. 

For  the  proof  of  Theorem  1,  see  Lemmas  3 to  6. 

It  should  be  noted  that  the  statement  of  Theorem  1 implies  that  the  relations  given  by 
(30)  to  (34)  also  include  values  which  are  not  optimal  returns.  This  is  so  because  of  necessity; 
the  dynamic-programming  solution  generates  negative  infinite  returns  for  some  values  of  X (see 
Ref.  Ill),  i.e.,  some  values  of  £S(X)  are  not  defined.  The  next  two  lemmas  deal  with  this  situa- 
tion. 


LEMMA  7:  If 


v,  < 


X — (A,_i  + 1) 


-I-  1,  As_i  < X < As,  then  cs  > As_i  + 1. 


(35) 


PROOF:  By  hypothesis,  we  have  that 

X — (Aj_i  + 1) 


v,  < 


+ 1. 


Since  [a]  < a,  we  can  remove  the  integer  requirement  of 

X — (As_i  + 1) 


in  (35)  and  strengthen  the  inequality.  Hence  we  have 

X — (AS_|  + 1) 

(36)  v,  < f-1 


+ 1. 


Since  c,  > 0,  we  can  rewrite  (36)  as 

(37)  X + c,  - (Aj_|  + 1)  > c,v, 
By  definition,  when  As_|  < X < A,  we  know  that 

(38)  vs  - u,  or  X < c,v,  + c,. 
The  inequalities  (37)  and  relations  (38)  together  imply  that 

(39)  c,  - (A,_i  + 1)  > 0. 


<L 
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Rewriting  (39)  we  have 


which  was  to  be  shown. 


c,  > A j-i  + 1. 


LEMMA  8:  If  c,  > A,_,  + 1,  then  jf,(X)  - for  X - As_,  + 1 + pc ,,  p = 0,  1 

and  x *(X)  is  undefined. 

PROOF:  The  dynamic  programming  recursion  relations  are 

gs(X)  - max  [ctx,  + £ ,(x  - csx,)J.  s - 2,  3 n. 

0<x,  <■,(*> 

Consider  g,(K),  X — As_|  + 1 + pc„  p - 0.  1 We  then  have 

+ 1 + pc, ) - maxlO  + g,_,(Av_t  + 1 + pc,),  c , + gs^,(As_t  + 1-1 -pc,  - c,), 

....  cv6,(X)  + #,_|(A  s_,  + 1 + pc,-  c,8,(X))], 

(40)  where  6,(X)  - min 

Consider  the  terms  in  the  brackets  in  (40).  The  first  term  exceeds  the  limit  As_!  of  the  func- 
tion gj-i(-)  for  all  values  of  p.  Since  c,  > A,_)  + 1 by  hypothesis,  ail  arguments  of  g,-\()  in 
subsequent  terms  will  either  exceed  the  limit  A, or  be  undefined  if  the  argument  is  negative. 
This  will  depend  upon  the  relative  magnitudes  of  p and  x,.  However,  in  no  case  can  the  value 
be  both  less  than  or  equal  to  A,.,  and  nonnegative.  This  follows  because  the  argument  of  £s-i 
is 

(41)  A,_j  + 1 + c,(p  - xs) 

CASE  1,  p < xs: 

It  then  follows  that  p - x,  < 0.  Since  c,  > A,_,  + 1.  As_,  + 1 + c,(p  - xs)  < 0,  and 
negative  arguments  for  gs_,(-)  are  not  defined. 

CASE  2,  p - x 

It  then  follows  that  (41)  reduces  to  A,_j  + 1,  which  exceeds  the  limit  of  the  arguments 
for  g, _,(■). 


A,.,  + I + pc. 


i 


1 


i 


Case  3,  p > x,: 

c,(p  - x,)  > 0. 

Therefore  A,_t  + 1 + c,(p  - x,)  exceeds  the  limit  A,_i  allowable  for  g,-|(  ). 
This  completes  the  proof. 


The  import  of  Lemmas  7 and 
what  was  the  significance  of  v,  < 


8 is  to  settle  the 
X - (A, -|  + 1) 

c, 


question  raised  at  the  end  of  Lemma  6 on 
+ 1 for  A,_|  < X < A,.  By  Lemma  7, 


this  fact  implies  that  c,  > A,_|  + 1,  and  Lemma  8 tells  us  that  if  this  latter  fact  is  true  then 
x,’(X)  is  undefined.  Hence,  the  backwards  recursion  may  be  discontinued  for  the  set  of  values 
x„  being  tested  with  the  current  value  of  zk,  and  the  next  set  of  x,  values  may  be  tested. 


1 
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The  significance  of  Theorem  1 and  Lemmas  7 and  8 is  that  the  entire  backwards  recursion 

n 

for  the  optimal  values  x’(zk).  xB*_,  (z*  - c„x „*) x,'(z*  - £ ‘"s-’O  maV  be  calculated,  given 

any  value  of  zk,  from  the  equations  (30)  to  (34)  in  Theorem  1 without  ever  carrying  out  the 
forward  calculation  and  storage  of  lengthy  tables.  Furthermore,  Lemmas  7 and  8 tell  us  that  in 
the  course  of  the  calculation  if  we  encounter  for  some  s - q that  cq  > A,_i  + 1,  we  may  ter- 
minate the  calculation  for  the  current  zk  and  proceed  to  begin  again  with  zk  - 1. 

In  the  next  section  we  shall  give  two  examples  of  the  use  of  the  theory  we  have  just 
developed. 

5.  HYPERPLANE  SEARCH  ALGORITHM  - TWO  EXAMPLES 

Example  1:  Max  z — x,  + 3x2  + 4x3  + 6x4, 

2x|  + 3x2  + 6x3  + 4x4  ^ 23, 

Sx,  + 4x2  + 2x3  + x4  < 20, 

(41)  x j > 0,  integer,  j - 1,  2,  3,4. 

First  we  solve  (41)  as  a linear-programming  problem,  ignoring  the  integrality  requirement.  If 
we  do  so,  we  obtain  the  solution  x[  - 0,  x2  - 2.056,  x3  - 1.111  and  z*  -9.5.  Hence  z°  < 9. 
Let  us  now  apply  the  hyperplane  search  algorithm  equations  of  Theorem  1 . 

It  is  readily  seen  from  the  constraints  of  (41)  that  the  variables  are  bounded  as  follows: 

0 < x,  < 7,  0 < x2  ^ 2,  0 < x3  «£  2,  and  0 ^ x4  < 3. 

However,  if  we  solve  a linear  programming  problem  in  which  we  maximize  each  variable  in 
turn,  we  find  the  following  closer  bounds: 

0 < x,  < 5,  0 < x2  < 2,  0 ^ x3  < 1,  and  0 < x4  < 0. 

Therefore,  we  have 

A, -5,  A2  — 11,  A3  - 14,  and  A4  - 14. 

We  know  from  the  bounds  that  x4*(9)  - 0.  For  k - 9 we  have  that  c3  - 3 < 9 < 14  - A3. 
Therefore,  we  have  that  x3*(9)  - 0,  1.  Since  the  LP  solution  gave  x3  - 1.111,  we  try  x3  - 1. 
Then  \ - 6,  and  c2-3<6<lt-A2.  Hence  we  have  that  x2(6)  - 0,  1,  2.  Since  the  LP 
solution  had  x2  - 2.056,  we  try  x2  - 2,  which  leaves  X - 0 and  x,  - 0.  We  now  have  as  a 
candidate  solution  x,  - 0,  x2  - 2,  x3  - 1,  and  x4  - 0.  We  see  that  this  solution  does  satisfy 
the  constraints  of  (41)  and  hence  the  optimal  solution  to  (41)  is: 

x,*  - 0,  x2  - 2,  x3  - 1,  x4  - 9,  and  z*  - 9. 

If  we  wished,  we  could  check  to  see  if  there  are  any  other  optimal  solutions  by  checking  the 
other  possibilities.  In  this  problem  the  truncated  value  of  the  linear-programming  problem  gave 
the  value  of  z for  the  optimal  hyperplane. 

In  general,  one  may  have  to  reduce  z°  several  times.  However,  the  linear-programming  solu- 
tion for  the  objective  Junction  is  usually  close,  and  frequently  the  values  of  xj  are  reasonably 
close. 
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Example  2: 

Max  z = 10x,  + 8x2  + 7x3, 

8x)  + 7x2  + 5x3  < 34, 
x,  + x2  + x3  < 6, 

(42)  X|,  x2,  x3  ^ 0,  integer. 

The  linear  programming  solution  is  x,  - 1.333,  x3  = 4.667,  z =-  46.  Because  c,  * 1,  we  con- 
vert (42)  to  the  following: 

Max  z = x0  -I-  10xt  4-  8x2  + 7x3, 

8x!  + 7x2  + 5x3  < 34, 

X!  + x2  + x3  < 6, 

X|,  x2,  x3  > 0,  integer, 

(43)  x0  = 0 
The  following  bounds  are  easily  derived: 

0 ^ x0  < 0,  0 ^ x,  < 4,  0 < x2  < 4,  and  0 < x3  < 6. 

We  then  calculate: 


^0 

x3‘(46)  - 0.  1,  2 


0,  A,  - 40,  A2  - 72,  A3  = 114,  z 


min 


6, 


46; 

0,  1,  2,  3,  4,  5,  6. 


We  give  a sample  calculation  for  x3  - 0.  If  this  is  the  case  then  A - 46  and 


46-41 


8 


+ 1,  . . . , 4 — 1,  2,  3,  4.  If  Xj’  — 1,  then  A = 38  and  x,* 


38-1 


10 


3 - 4,  3.  Since  4 > 3,  this  corresponds  to  #,(38)  - -°°.  If  x2’  - 2,  then  A 
30-1  ’ 

*\  ~ 


10 


+ 1 

= 30  and 


+ 1.  . . . , 3 = 3.  This  gives  a potential  solution  x,  - 3,  x2  - 2,  x3  = 0.  How- 


ever, this  violates  one  of  the  constraints  of  (42).  If  x2’  — 3,  there  is  no  solution,  since  g\(22) 
— — °o,  and  if  x'2  — 4,  there  is  again  no  solution.  Repeating  this  entire  process  for  the  remain- 
ing values  of  x3  will  also  yield  similar  results. 


We  next  try  zk  — 45.  We  again  find 

x3"(45)  - 0,  1,  2,  3,  4,  5,  6. 


Noting  that  x3  in  the  linear-programming  solution  was  between  4 and  5,  a good  computational 
strategy  is  to  first  try  6 rather  than  zero.  If  xj  - 6,  then  A - 3.  This  yields  xj  - 0 and  xj 
nonexistent.  We  next  try  x3’  - 5.  This  yields  A - 10  and  xj  - 0,  1.  If  xj  - 0,  then  A - 10 

1 + 1,  •••  . 1-1.  We  have  a potential  solution  x,"  - 1,  xj  - 0,  and  xj  — 5. 


and  xj  — 


10 


This  is  found  to  satisfy  the  constraints  of  (42)  and  hence  we  have  found  the  optimal  solution. 


6.  COMPUTATIONAL  RESULTS 

In  order  to  gain  some  insight  into  the  computational  feasibility  of  the  method  developed 
in  this  paper,  an  experimental  code  was  written  to  solve  problems  with  randomly  generated 
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data.  A program  to  generate  problems  such  that  (1)  always  has  a feasible  noninteger  solution 
was  developed.  The  matrix  A had  full  density.  This  averaged  about  90%  dense  for  all  matrices 
generated  because  of  some  random  generation  of  zeros.  The  range  of  values  for  a„,  the  ele- 
ments of  A,  was  0 ^ a,,  < 10  for  all  the  problems  generated.  For  the  majority  of  problems 
tested,  the  values  of  cn  the  objective  function  coefficients,  were  in  the  range  / ^ c,  < 4. 
These  were  also  randomly  generated.  On  the  15  x 50  problem,  coefficients  in  the  range  0 ^ c, 
< 50  were  used.  However,  only  ten  of  the  coefficients  were  positive.  The  values  of  the 
coefficients  for  bn  the  components  of  the  requirements  vector  b , were  computed  as  follows.  A 
set  of  values  of  a "feasible  point",  x,  was  randomly  generated  in  the  range  (0,  4).  If  xn  are  the 
components  of  x, , then 

b,  — ^ On  Xfj  + 1 

/ 

By  this  means  the  generation  of  a problem  with  a feasible  solution  was  guaranteed. 

In  the  implementation  of  the  algorithm,  step  2,  the  determination  of  lower  and  upper 
bounds,  was  carried  out  using  equation  (5),  i.e.,  a simplex  calculation  for  the  larger  problems. 
For  smaller  problems,  a cruder  method  for  bound  determination  was  used.  All  calculations 
were  done  on  a CDC  CYBER  70,  Model  72,  which  is  a moderate  speed  computer.  The  results 
are  given  in  Table  1. 


TABLE  1 — Computational  Results 


m 

n 

No.  of 
Problems 

Mean  Time 
(S) 

Least 

Time 

Greatest 

Time 

Bounds  by 
Simplex 

3 

10 

10 

0.2 

0.05 

0.7 

No 

4 

10 

10 

5.7 

0.08 

28.6 

No 

5 

10 

10 

22.4 

0.08 

118.4 

No 

5 

15 

10 

14.8 

1.3 

89.8 

No 

5 

20 

1 

0.5 

— 

— 

No 

3 

15 

5 

0.9 

0.09 

3.0 

No 

4 

15 

10 

13.2 

0.2 

50.3 

No 

4 

20 

3 

32.5 

0.1 

97.2 

No 

5 

16 

1 

0.1 

— 

— 

No 

5 

21 

1 

1.0 

— 

— 

No 

4 

24 

1 

5.6 

— 

— 

No 

4 

28 

1 

0.2 

— 

— 

No 

5 

32 

1 

0.2 

— 

— 

No 

8 

10 

20 

2.18 

2.0 

3.0 

Yes 

10 

20 

25 

12.5 

5.0 

71.0 

Yes 

10 

25 

5 

140.8 

11.0 

639.0 

Yes 

10 

30 

7 

207.1 

19.0 

1284.0 

Yes 

12 

33 

2 

26.5 

21.0 

32 

Yes 

15 

40 

4 

342.8 

54.0 

782.0 

Yes 

15 

50 

1 

218.0 

- 

Yes 

The  results  of  Table  I indicate  that  the  method  has  been  used  successfully  on  problems  of 
relatively  moderate  size.  Further  tests  on  larger  problems  have  not  been  made  for  a number  of 
reasons.  First,  program  and  storage  optimization  would  significantly  alter  the  times  presented 
in  Table  1.  More  importantly,  the  wide  variation  in  times  found  for  problems  of  the  same  size 
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is  probably  due  to  the  nature  of  the  problems,  i.e.,  a matrix  A with  high  density  and  with  ran- 
domly generated  coefficient  entries.  Real-world  problems  would  be  much  more  sparse  and 
highly  structured.  Hence,  no  definite  conclusions  can  be  drawn  from  Table  1 about  how  the 
method  would  operate  on  such  problems.  Unfortunately,  such  problems  are  not  available  to 
the  authors  for  testing.  The  results  of  Table  1 are  sufficiently  encouraging,  however,  that  this 
method  should  be  explored  further  on  real  problems. 
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ABSTRACT 

Infinite-horizon,  countable-state,  continuous-time  Markovian  decision 
models  are  solved  by  formulating  as  a pair  of  infinite  linear-programming  prob- 
lems. Expected  discounted  and  average  returns  are  considered  as  criterion 
functions.  For  both  criterion  functions,  the  existence  of  deterministic  optimal 
stationary  policies  is  established  by  solving  the  associated  infinite  linear- 
programming  problems.  Computational  procedures  for  finite  state  and  action 
sets  are  discussed  by  considering  associated  finite  linear-programming  problems. 


1.  INTRODUCTION 

In  this  paper,  we  develop  a methodology  that  can  be  used  to  analyze  stochastic  dynamic 
systems,  such  as  health,  transportation,  educational,  economic,  and  production  systems.  These 
systems  change  continuously,  due  to  changes  in  the  environment  in  which  they  operate.  Some 
of  the  factors  that  influence  these  systems  are  controllable  and  others  are  not,  hence  the 
behavior  of  the  systems  cannot  be  predicted  completely.  This  makes  it  appropriate  to  treat  the 
systems  as  stochastic  rather  than  deterministic.  Since  some  of  the  factors  that  influence  a sys- 
tem are  controllable,  our  object  is  to  determine  how  to  adjust  these  controllable  factors,  so  that 
the  system  operates  satisfactorily.  We  assume  that  the  possible  states  of  the  system  are  count- 
able and  that  the  available  actions  are  finite;  they  are  denoted,  respectively  by  5 and  A.  The 
changes  in  the  system  are  governed  by  a set  of  known  numbers  q^ (a)  (/',  j €S,  a 6 A),  called 
the  transition  rates.  The  transition  rates  may  be  interpreted  as  probabilities;  that  is,  if  at  time  t 
the  system  is  in  state  i,  i€S,  then  the  approximate  probability  that  the  system  is  in  state  j ^ i 
after  time  8 1 is  given  by  ql/(a)6t,  and  the  probability  that  it  remains  in  the  same  state  is 
[1  - T.i^,qtl(a)ht].  The  system  may  be  observed  at  any  time  and  classified  into  one  of  the 
possible  states  i,  i£S.  If  the  system  is  in  state  / 65 and  an  action  a€A  is  taken,  then  the  sys- 
tem receives  a bounded  return  r ( /,  a ) per  unit  time  and  moves  to  the  new  state  J€S  governed 
by  the  transition  rates  </„(a).  Under  certain  general  conditions  on  {<7, /a)),  the  trtuectory  of  the 
stochastic  dynamic  system  can  be  represented  by  a Markov  process.  In  view  of  this,  we  use  the 
words  system  and  process  synonymously. 

We  are  concerned  with  the  optimal  control  of  the  type  of  stochastic  systems  described 
above.  For  controlling  such  systems  we  need  a rule  that  prescribes  the  action  to  be  taken;  this 
rule,  denoted  by  ir,  is  called  a policy.  It  is  assumed  that  the  policy  ir  is  specified  by  a family  of 
measurable  functions  (d,a  (f ) ),  where,  for  each  / €S 

£</,„  <•)-!.  and  d,a  (•)  ^ 0,a  € A. 


(1.1) 


432 


P.  KAKUMANU 


Thus  the  function  d,a(t)  may  be  interpreted  as  the  probability  of  taking  an  action  a,  a €/t,  at 
time  /,  given  that  the  current  state  of  the  system  is  /€S.  A policy  ir  is  called  Markovian  if 
d,a(t)  « 1 or  0,  and  it  is  called  stationary  if  d,a(  ) is  independent  of  / and  satisfies  (1.1).  A sta- 
tionary policy  is  called  a deterministic  stationary  policy  if,  for  each  i,  d,a  = 1 or  0,  /€S,  a€A. 
That  is,  a deterministic  stationary  policy  specifies  a single  action  depending  upon  the  state  of 
the  processes  but  independent  of  time  /. 

When  a policy  it  is  applied,  the  transition  rate  from  a state  / to  a state  j is  given  by 

(1.2)  it)  - £0„(a)  d,„{t),  /',  j € S. 

a 

Let  Q(t,ir)  be  the  transition  rate  matrix  whose  (/J)"' element  is  given  by  (1.2).  For  each  /€S, 
the  transition  rates  are  assumed  to  satisfy: 

(1.3)  T,q,/(t.  ir)  = 0.  ?„(/.ir)  > 0,  j * i,  and 

/ 

(1.4)  0 < | q„  (/,  it)  | ^ M < °°. 

Under  these  assumptions,  it  was  shown  in  Ref.  [13]  that,  for  any  given  Markov  policy  ir,  a 
unique  transition  probability  matrix 

(1.5)  F(s,t,  it)  = \f,j  (s.t.n),  i,  j € S) 

exists,  and  that  for  almost  all  t > s it  satisfies  the  Kolmogorov  differential  equations: 

(1.6)  l’  ^ = F(s,t,ir)  Q (/. ir),  with  F (s.  s,  ir)  = I. 

3/ 

If  it  is  a stationary  policy,  then  the  transition  rate  matrix  Q is  independent  of  /,  and  the 
transition  probability  matrix  ^defines  a time-homogeneous  Markov  process  x(t,  n). 

The  expected  rate  of  return  out  of  a state  /,  when  the  Markov  policy  n is  given  by 

(1.7)  /(/,  t.  it)  — £r(i.  a)d,a  (/),  / ^ 0,  / € 5. 

a 

Since  the  rate  of  return  r(i,a)  is  uniformly  bounded,  it  is  obvious  from  (1.7)  that  /•(/,  /,  ir)  is 
uniformly  bounded  in  / and  ir.  For  any  Markov  policy  ir,  using  the  above  definitions  and  nota- 
tions, we  define  two  economic  criterion  functions.  The  first  one,  the  total  expected 
discounted-return  function,  which  is  given  by: 

(1.8)  ST  (I,  a,  ir)  - f e-a'Z/(/  (o,  /,  ir)  r (J,  t,  it)  dt.  i£S, 

•'0  j 

where  a > 0,  is  called  the  discount  factor.  The  other,  the  long-run  expected  average-return 
function,  is  defined  by: 

r T 

(1.9)  4>(/,  ir)  - lim  T~'  J T/(.  (o,  t,  n)  r (J,  I,  n)  dt,  itS. 

r— oo  •'o  j 

If  the  limit  in  (1.9)  does  not  exist  for  any  / €S,  then  we  set  <*>(/.  ir)  - for  that  /. 

A policy  ir*  is  called  an  a-discounted  optimal  policy  if  i ]/(/.  «,  ir*)  > </»(/,  a, ir),  /€S. 
and  it  is  called  an  average  optimal  policy  if  4>(/,  ir*)  ^ 4>  (/,  ir).  i€S,  where  ir  is  any  Markov 
policy. 

The  object  of  this  paper  is  to  show  the  existence  of  an  optimal  deterministic  stationary 
policy  for  the  discounted  and  the  average-return  case,  by  considering  a separate  pair  of  infinite 
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linear-programming  problems.  A linear-programming  problem  is  called  an  infinite  linear- 
programming  problem  if  the  number  of  variables  and  constraints  are  countably  infinite  [6). 

The  methodology  developed  here  is  important  mainly  for  two  cases  which  are  currently 
under  investigation  by  the  author.  In  the  first  case,  using  the  results  developed  by  Fox  [9],  and 
Gustafson  and  Kortanek  1 10],  we  can  obtain  an  approximate  optimal  policy,  along  with  the 
corresponding  return  for  the  criterion  functions  (1.8)  and  (1.9)  by  considering  a series  of  finite 
linear-programming  problems.  It  is  also  possible  to  estimate  the  error  between  the  approximate 
and  optimal  solutions,  and  the  convergence  rate.  In  the  second  case,  that  of  the  discounted 
model,  the  methodology  can  be  used  to  parametrize  the  discount  factor.  This  enables  us  to 
identify  the  interval  in  which  an  optimal  policy  is  invariant.  The  parametric  analysis  of  the 
discount  factor  will  be,  in  a certain  sense,  an  extension  of  the  results  given  by  Denardo  [3], 
Howard  (Ref.  [11],  p.  88),  Miller  and  Veinott  [16],  Mine  and  Osaki  (Ref.  [18],  p.  22),  and 
Veinott  [22]. 

Semi-Markovian  or  Markov  renewal  decision  processes  are  also  used  to  study  stochastic 
systems  (see  Ross,  Ref.  [20],  p.  156).  In  this  case,  the  decisions  are  made  only  at  the  instant 
of  transitions,  whereas  in  the  continuous-time  Markovian  decisions  models  the  decision  are 
made  at  any  time.  As  pointed  out  by  Doshi  [5],  the  optimal  policy  we  obtain  using  semi- 
Markovian  decisions  might  result  in  suboptimal  operation  of  the  system.  In  addition  to  this, 
with  continuous-time  Markovian  decision  models  we  need  less  data  than  we  do  with  semi- 
Markovian  decision  models  to  analyze  a stochastic  system.  For  these  reasons,  continuous-time 
Markovian  decision  models  are  preferable  to  semi-Markovian  decision  models  for  studying  cer- 
tain stochastic  systems. 

In  Section  2,  the  existence  of  a-discounted  optimal  deterministic  stationary  policies  is 
proved  for  all  Markov  policies.  In  Section  3,  the  analogous  results  for  the  average-return  cri- 
terion function  is  established.  Finally,  in  Section  4,  the  computational  aspects  of  the  results 
obtained  in  the  earlier  sections  are  discussed  when  the  state  space  is  finite. 

2.  COUNTABLE-STATE  DISCOUNTED-RETURN  MODEL 

In  this  section,  the  following  results  are  established  by  formulating  the  expected 
discounted-return  model  as  a pair  of  infinite  linear  programs  called  primal  and  dual  problems. 
Primal  and  dual  problems  always  have  bounded  feasible  solutions.  There  is  one-to-one 
correspondence  between  the  set  of  dual  feasible  solutions  and  the  set  of  stationary  policies. 
Both  problems  have  bounded  optimal  feasible  solutions  such  that  the  corresponding  values  of 
their  objective  functions  are  equal.  At  least  one  dual  optimal  feasible  solution  is  capable  of 
interpretation  as  an  a-discounted  optimal  deterministic  stationary  policy.  Finally,  a sufficient 
condition  is  given  which  will  guarantee  that  every  dual  optimal  feasible  solution  will  yield  an 
a-discounted  optimal  deterministic  stationary  policy. 

The  methodology  given  in  this  section  provides  the  means  of  obtaining  an  approximate 
optimal  policy  for  the  criterion  function  (1.8),  and  of  conducting  parametric  analysis  of  the 
discount  factor  a.  These  could  not  have  been  done  by  using  the  results  given  in  Ref.  [14]  and 
the  finite  state  policy  interaction  methods  because  of  computational  difficulty.  The  results 
obtained  here  are  a generalization  of  discrete-time  results  proved  by  Evans  [8],  Some  tech- 
niques given  by  him  will  be  used  to  establish  our  results. 
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i 


i 


Let  {y(;  y€S)  be  a set  of  positive  number  such  that  £y,  - 1,  and  g be  a vector  defined 

i 

on  S with  bounded  components.  Consider  the  following  pair  of  infinite  programs: 


PRIMAL  I 

DUAL  I 

Inf  £y(  g, 

sup  I I »>,„  rO.  a) 

i 

subject  to 

a 8,  -!«,/(«)  8,  > r(i.a). 

subject  to 

I I w,a  8„  “ <7,/(fl))  ” Yr 

i€S,  a 6/4. 

wia  > 0,  jtS,  at  A. 

LEMMA  2.1  The  primal  always  has  a bounded  feasible  solution. 

PROOF:  Let  0(/,  air*)  — sup  * (/',  a,  it),  /€S.  Then,  it  is  known  [14]  that 

It 

*(/,  a.  ir  *) , /'  € S,  is  the  unique  solution  of  the  functional  equation 
ag,  - max0  |r(/',a)  + £<7,7(0) 

Taking  g,  - </»(/,  a,  ir*),  /€S,  satisfies  the  primal  constraints.  Hence,  {*(»',  a,  ir*))  is  a primal 
feasible  solution.  The  value  of  the  objective  function  is  given  by 

£y,  |*(/,  a,  ir  *)  | - £y,  !/0~  e-'  I/,7  0,  *•)  r 0.  **)  dt\ 

I J i 

<(L>Hhp|ra,r*)lI 

Since  Ty,  - 1,  the  r(y,  ir*)  are  uniformly  bounded,  and  a > 0,  £y,  *(/',  a.  w*)  is  a finite 

, 

number. 


LEMMA  2.2  The  dual  has  a bounded  feasible  solution. 


PROOF:  Let  ir  be  any  stationary  policy  specified  by  {</,„).  Define 

(2.1)  u,a  “ !>/  J0  ff)  d'°  dt-  i £ S-  a £ A. 

Now  we  will  show  that  {«,a)is  a bounded  dual  feasible  solution.  It  is  obvious  that  u,a  ^ 0,  / €S, 
a€/<.  Substituting  u,a  from  (2.1)  for  w,a  in  the  left  hand  side  of  the  dual  constra.nl,  we  arrive 
at: 

II  [«  - q ,,(*)]  £y,  Jf  e~°' fu  U,  ir)  d,a  dt,  j € S. 

i a I 

Since  the  series  converges  absolutely,  we  obtain  after  some  simplification 

£y,  J0~  e~a'  |a  /„  (/.  ir)  - £/„  0.  ir)  q,f  (ir)  j dt.  j 6 S. 

Using  (1.6)  and  integrating  by  parts,  we  can  show  that  this  is  equivalent  to  y/,  j€S.  Substitut- 
ing the  value  of  {uj  from  (2.1)  in  the  dual  objective  function  for  w,a,  we  obtain  after  some 
simplification 

(2.2)  I IM*  '■('•  fl)  ” I M ff)- 

/ a I 
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Since  y,  - 1,  and  ■/»(/,  a.  *r)  is  uniformly  bounded  in  /.  a.  and  it,  the  righthand  side  is 
finite.  Hence,  [u,a)  is  a bounded  dual  feasible  solution. 

THEOREM  2.1:  There  is  one-to-one  correspondence  between  the  set  of  dual  feasible 
solutions  and  the  set  of  stationary  policies. 

PROOF:  Let  {h^J  be  any  dual  feasible  solution.  First  we  will  show  £ wia  > 0,  jtS. 

a 

The  dual  constraints  may  be  written  as 

(2  3)  [L  *«.)  (“  “ 9//(o)  1 - £ £ w,a  q,i(a)  - y ,,  jtS. 

I a I a 

Since  y/  > 0,  jtS,  and  {^(/(a)}  satisfies  (1.3),  we  have 

(2  4)  £ wia  > 0. 

a 

Define 

(2.5)  “ •,  , itS,  o t A. 

2*  w,h 
b 

By  definition,  the  set  [d,a)  defines  a stationary  policy,  say  8.  If  (wj  is  the  set  of  values  given 
by  (2.1)  when  the  policy  8 is  applied,  as  shown  in  Lemma  2.2,  [u,a ) is  a dual  feasible  solution. 
If 

(2  6)  ula  - w,a,  itS,  a 6/4. 

then  we  can  conclude  that  there  is  one-to-one  correspondence  between  the  set  of  feasible  solu- 
tions and  the  set  of  stationary  policies.  Now  we  will  show  that  (2.6)  is  true. 

Let  w,  - £ wia,  itS,  and  W,  and  y represent  the  row  vectors  whose  i'h  elements  are 

a 

given  by  w,  and  y„  itS,  respectively.  Using  this  notation  and  (1.2),  we  may  write  2.3  as 

W[a!  - Q(8)]  - y, 

since  la/  - 0(8)1  is  the  resolvent  of  the  operator  0(8).  From  Dynkin  (Ref.  [7],  p.  24)  we 
obtain 

W-  f“  e~a’yF(t,  i)dt. 

The  /'*  component  of  H'is  given  by 

L “ Z?/  f,  *■"'//,  0.  8)  dl.  itS. 

a I u 

Using  (2.1)  and  (2.S)  we  obtain: 

!»»■  KS- 
a a 

From  this,  (2.1),  and  (2.3)  we  conclude  that  (2.6)  is  true. 

LEMMA  2.3:  If  |g,)  and  |wM)  are,  respectively,  primal  and  dual  feasible  solutions,  then 
(2-2)  L?/  */  > I r(i,  a). 

/ / a 
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Since  the  infinite  series  that  appear  in  both  primal  and  dual  are  absolutely  convergent,  the 
proof  of  the  lemma  is  similar  to  the  corresponding  result  in  finite  linear  programming  [21]. 

Since  the  primal  and  dual  problems  have  bounded  feasible  solutions,  the  optimal  solutions 
for  both  problems  will  always  exist.  In  the  following  theorem,  we  establish  that  there  is  no 
duality  gap,  that  is  that  the  values  of  the  objective  functions  for  the  corresponding  optimal 
feasible  solutions  are  equal.  Then,  using  the  results  developed  here,  we  establish  the  existence 
of  an  a-discounted  optimal  deterministic  stationary  policy. 

THEOREM  2.2.  There  is  no  duality  gap. 

PROOF:  Let  (tv,')  be  a dual  optimal  feasible  solution  and  w * be  the  corresponding  sta- 
tionary policy  obtained  from  |wj  through  (2.5).  Now,  following  the  argument  given  in 
Lemma  2.2,  we  obtain 

(2.8)  ZL  w'«  r(/-  a)  “ Zr-  * ('•  n *>■ 

/ a 

Since  -y,  > 0,  /€S,  and  the  same  it*  maximizes  <//(/.  a,  it)  over  all  stationary  policies  for 
every  / € S,  we  obtain 

a,  it*)  - sup  (i li(i.  a,  w)l,  i€S. 

n 

In  Lemma  2.1,  it  was  shown  that  g,  — i|»  (/,  a tr*),  i€S,  is  a primal  feasible  solution.  From 
(2.8)  we  conclude  that  (s,')  is  a primal  optimal  feasible  solution,  and  that  the  values  of  the  pri- 
mal and  dual  objective  functions  are  equal. 

THEOREM  2.3:  An  a-discounted  optimal  deterministic  stationary  policy  always  exists, 
and  there  is  at  least  one  dual  optimal  solution  that  will  give  this  policy. 

PROOF:  Let  {#,'}  be  the  optimal  primal  feasible  solution.  Consider  the  sets 

T i - - rU.aD  + (a)  g],  a , €^|,  KS, 

where  a^A  is  the  action  taken  when  the  state  of  the  processes  is  KS.  For  each  state  / € S it 
can  be  easily  shown  that  T,  is  not  an  empty  set  [14].  Let  us  define 

a,‘-  min  [l\a,  € T ,}, 

i 

1 if  a — a,. and 
0 otherwise. 

Then  a policy  a*can  be  obtained  from  {v,a}  using  (2.5),  and  it  is  clear  that  cr  is  a deterministic 
stationary  policy.  As  in  Theorem  2.1,  we  can  show  that  [v,')  is  a dual  feasible  solution.  Using 
(2.2)  and  Theorem  2.2,  we  may  write  the  dual  objective  function  as 

ZZM*«  fl> " Z y>* 

i a * 

Hence,  cr*  is  an  optimal  deterministic  stationary  policy,  and  (v,0)  is  the  optimal  dual  feasible 
solution  that  will  yield  this  policy. 

In  Theorem  2.6,  we  give  a sufficient  condition  that  will  guarantee  the  optimal  determinis- 
tic stationary  policy  for  every  dual  optimal  feasible  solution.  We  can  prove  the  following  two 
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theorems  easily  using  Lemma  2.3,  Theorem  2.2  and  the  duality  theory  of  finite  linear  program- 
ming. 

THEOREM  2.4:  The  necessary  and  sufficient  condition  that  any  primal  |jf,}  and  dual  { w,a) 
feasible  solutions  are  optimal  to  respective  problems  is  that 

Lyff,  “ LL  w<a  «)• 

i ' i a 

THEOREM  2.5:  Any  pair  of  primal  and  dual  (iv,J  feasible  solutions  is  optimal  to 
respective  problems  if  and  only  if 

w,a  |<*£,  - X<7(/  (a)j?,  - r(/,a)j  - 0.  /€S,  a£A, 
and 

L L W:a  “8„  - j “ 0.  / €S. 

THEOREM  2.6:  If  there  is  a primal  optimal  solution  |g,l  such  that 

(2.9)  ag,'-  r(i.  a)  + £ q„(a)g' 

i 

for  only  one  a€A  for  each  /€S,  then  every  dual  optimal  solution  yields  an  a-discounted 
optimal  deterministic  stationary  policy. 

PROOF:  Let  a'  €A  be  the  unique  a for  which  (2.9)  holds,  that  is 

(2-10)  ag*  - r(i,  a,')  + (a')g',  i£S. 

i 

For  any  other  a €A,  we  have 

(2.11)  ag,'  > r(i,  a)  + £<7,,(a)  g‘,  i€S. 

j 

Let  [w*,a]  be  any  dual  optimal  feasible  solution.  Then  from  (2.4)  we  have 

(2.12)  2>,;>o. ,€5- 

a 

From  Theorem  2.5,  (2.10),  and  (2.11),  for  each  /€S,  we  have 

(2.13)  w‘a  ^ 0 if  a — a*, and 

- 0 for  all  other  a€A. 

From  (2.12)  and  (2.13),  we  obtain  for  each  i€S 
w,‘a  > 0 if  a - a*,  and 


- 0 for  all  other  a€A. 
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Now  define 

d'm  - - 1 or  0.  itS,  at  A. 

L»,t, 

b 

Hence,  by  Theorem  2.1,  cr*is  an  a-discounted  optimal  deterministic  stationary  policy  and  the 
corresponding  optimal  return  is  give  by  g'  — i/i(/,  a,  tr*),  itS. 

3.  COUNTABLE-STATE  AVERAGE-RETURN  MODEL 

In  this  section,  by  considering  a pair  of  infinite  linear  programs  we  establish  the  existence 
of  an  average  optimal  deterministic  stationary  policy.  The  methodology  that  will  be  developed 
in  this  section  is  analogous  to  that  given  in  Section  2.  In  view  of  this,  we  will  briefly  indicate 
the  proof  for  the  main  results  and  the  rest  will  be  stated  without  proofs.  Consider  a pair  of 
infinite  linear  programs  associated  with  the  average-return  model: 


PRIMAL  II 

DUAL  II 

Inf  /i 

Sup  '(/.a) 

subject  to 

i a 

subject  to 

h ~ vi  > rU.a), 

ZI^  <hj(a)  - 0,  jts,  and 

i€S,  at  A. 

- 1. 

i o 

xia  ^0.  itS,  a € A. 

Throughout  this  section  we  need  the  following  assumption: 

ASSUMPTION  3.1:  For  each  deterministic  stationary  policy  ir,  the  resulting  Markov  pro- 
cess x(t.  it)  is  positive  recurrent  with  only  one  recurrent  class. 

THEOREM  3.1:  Primal  and  dual  problems  have  bounded  feasible  solutions. 

PROOF:  The  existence  of  bounded  primal  feasible  solutions  is  proved  by  use  of  the  argu- 
ment given  in  Lemma  2.1,  and  Theorem  3.1  of  Ref.  [15].  The  existence  of  dual  feasible  solu- 
tions can  be  shown  as  follow.  Let  tr  be  any  stationary  policy  specified  by  the  set  {</,„).  Con- 
sider 

(3.1)  y,a  - lim  -7  / A U d»  dt'  /€S- 

It  is  obvious  that  LyJ  satisfies  nonnegative  restrictions  and  that  £ - 1.  Substituting  the 

i a 

value  of  y,a  from  (3.1)  in  the  left-hand  side  of  the  first  dual  constraint,  after  some 
simplification  we  arrive  at 

lim^  y /0  |LA  (/.  n)qu  (w)J  dt,  jtS. 

We  can  show  that  this  is  equal  to  zero  by  using  (1.6)  and  integrating  by  parts.  Since  r(i,  a)  is 
uniformly  bounded  in  i and  a,  it  follows  that  £ £.V/D  r^‘  a ) 's  Hence,  {>J  is  a 

/ a 

bounded  dual  feasible  solution. 


r 


1 


f 
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THEOREM  3.2: 
solutions  and  the  set  of 

PROOF:  Let  (xj 


(3.2) 


There  is  one-to-one  correspondence  between  the  set  of  dual 
stationary  policies. 

be  any  dual  feasible  solution.  As  in  Theorem  2.1,  we  can  show 
d„  - iSS.  at  A. 

i*xtb 


feasible 

that 


b 

defines  the  probability  distribution  on  A for  each  itS.  This  shows  that  the  dual  feasible  solu- 
tion (x,J  yields  a stationary  policy,  say  tt. 


Let  p,(ir),  itS , be  the  steady-state  probabilities  of  processes  when  the  stationary  policy  n 
is  used.  Under  Assumption  3.1  p,(ir),itS , satisfies  the  following  conditions: 

(3.3)  £p,(tt)  <7,;  (tt)  - 0,  itS, 

i 

£p,  (ir)  = 1,  and  p,(rr)  > 0,  itS. 

I 

Using  (1.2),  (3.2),  (3.3),  and  dual  feasible  solution  |x,J,  we  obtain 

(3.4)  Ix/a  - p,  (rr),  itS. 


Noting  that  p,  (ir) 


(3.5) 


r T 

lim  T~'  I (7.  t r)  dt , itS , and  from  (3.1)  and  (3.4),  we  have 

r— °°  •'o 

- Ljv  /6S- 

a a 


From  (3.2)  we  have,  for  »€Sand  at  A,  y,a  - ^y,h  dia  - ^x,b  d,a  - x,„. 

A A 

This  shows  that  the  stationary  policy  n yields  a set  of  numbers  b^)  from  (3.1),  which  are 
equal  to  the  dual  feasible  solution  that  is  used  to  obtain  n,  which  proves  the  theorem. 

For  the  primal  and  dual  problems  considered  in  this  section,  the  results  corresponding  to 
those  of  Lemma  2.3  and  Theorems  2.4  and  2.5  can  be  proved  by  use  of  the  arguments  given  in 
Ref.  [21]  for  finite  linear  programs.  In  the  following  theorem,  we  establish  the  existence  of 
optimal  primal  and  dual  feasible  solutions  with  no  duality  gap  and  the  existence  of  an  average 
optimal  deterministic  stationary  policy.  The  results  correspond  to  Theorems  2.2  and  2.3. 

THEOREM  3.3:  Both  the  primal  and  dual  problems  have  bounded  optimal  feasible  solu- 
tions with 

(3.6)  Inf  h — sup  IIvU  «). 

I a 

and  there  exist’s  an  average  optimal  deterministic  stationary  policy. 

PROOF:  It  was  shown  in  Theorem  3.1  that  a bounded  primal  feasible  solution  (/i*.  v‘) 
exists.  Let 

T,  - |/|/i*  - r (/,  a, ) + £<7,,  (a,)  v‘.  It 
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where  a,  is  the  action  taken  when  the  process  is  in  state  KS  and  at  least  one  such  a,  exists. 
Let  a,’  - minfa,  | KT,},  i£S,  and  o-*be  a policy  that  prescribes  the  action  a,"  when  the  pro- 
cess is  in  state  i€S.  It  is  clear  that  <r*is  a deterministic  stationary  policy  and  satisfies. 

h*  - r(i,  cr)  + (o'  *)  v’,  i£S. 

J 

Then,  from  Theorem  3.3  of  Ref.  [15],  we  have 

(3.7)  h*  — <r *),  /€5. 


Let  {y,’J  be  the  solution  obtained  from  (3.1)  when  the  policy  <r‘is  used.  In  Theorem  3.1 
it  was  shown  that  !>>,*}  is  a dual  bounded  feasible  solution.  The  corresponding  dual  objective 
function  is 

'L'Ly’a  rd-  «)■ 

/ a 

Substituting  the  value  for  y‘a  from  (3.1), 

f*im  t~'  X ('• d*ia  H a^’ 

i a 1 r~”  * 

where  [d'J  corresponds  to  <r*.  Since  all  the  terms  are  positive,  the  limit  and  the  summation 
signs  can  be  interchanged.  Using  (1.7)  and  (1.9),  after  some  simplification  we  arrive  at 

(3.8)  '(/.  «)  “ <D(A  «»••).  KS. 

I a 

We  can  show  from  (3.7)  and  (3.8)  that  ( h *,  v/}  is  a primal  optimal  solution  and  {>,0)  is  a dual 
optimal  feasible  solution  as  the  solutions  attain  their  lower  and  upper  bounds  respectively.  The 
policy  <r*is  an  average  optimal  deterministic  stationary  policy. 

Under  certain  conditions  similar  to  those  given  in  Theorem  2.6,  it  can  be  proved  that 
every  dual  optimal  feasible  solution  is  capable  of  interpretation  as  an  average  optimal  deter- 
ministic stationary  policy.  All  the  results  established  in  this  section  can  be  shown  easily  when 
the  time  parameter  / is  discrete  instead  of  continuous. 

4.  FINITE-STATE:  CASE  COMPUTATIONAL  PROCEDURES 

In  this  section  we  assume  that  the  state  and  action  sets  are  finite;  they  are  denoted  respec- 
tively by  S - [1,2 m ] and  A - (1,2 Z,),  where  m and  L are  finite  positive  integers. 

The  summations  over  ;,  j,  and  k vary  from  1 to  m,  and  over  a and  b from  1 to  L.  When  the 
state  and  action  sets  are  finite,  the  primal  and  dual  problems  given  in  Sections  2 and  3 can  be 
written  as  standard  linear  programs. 

The  main  purpose  of  discussing  the  finite  state  models  is  to  show  their  use  in  obtaining 
the  solutions  for  the  infinite  linear-programming  problems  discussed  in  Sections  2 and  3.  This 
will  be  done  by  solving  a series  of  finite-state  problems  whose  solution  is  known  to  converge  to 
that  of  the  countably  infinite  state  problem  [9],  For  a finite  state  space,  Denardo  [2],  Howard 
[12],  Mine  and  Osaki  [18],  and  Osaki  and  Mine  [19]  have  formulated  Markovian  renewal  pro- 
gram problems  as  finite  linear-programming  problems.  For  the  reasons  stated  in  Section  1, 
sometimes  it  is  preferable  to  study  the  decision  process  as  a continuous-time  Markovian  process 
rather  than  as  the  Markov  renewal  process. 

We  first  discuss  the  discounted-return  model.  Mine  and  Tabata  [17]  formulated  finite- 
state  continuous-time  Markovian  decision  models  as  standard  linear-programming  problems  by 
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using  various  transformations.  These  transformations  limit  the  use  of  linear-programming  tech- 
niques for  solving  the  continuous-time  Markovian  decision  models.  The  methodology 
developed  in  Section  2 will  allow  us  to  formulate  the  continuous-time  models  as  a pair  of  stan- 
dard linear  programs  without  the  help  of  any  type  of  transformation.  This  type  of  formulation 
is  very  important  in  obtaining  the  solution  to  continuous-time  Markovian  decision  models  by 
solving  the  associated  linear  programs: 


PRIMAL  III 

DUAL  III 

min 

max  XX  r(/',  a)wl(l 

/ 

subject  to 

«£<  ~ > r(i.a). 

i a 

subject  to 

2.aWia  - XX  W,a  Q.ila)  = y,.  jZS, 

itS.  ak/ i. 

W/a  ^ o.  yes,  a €/f. 

The  numbers  y t,  j€S,  are  strictly  postive  and  add  up  to  unity.  The  results  that  were 
proved  in  Section  2 hold  in  particular  when  the  state  space  is  finite.  We  now  show  that  every 
dual  optimal  solution  obtained  by  the  simplex  method  may  be  interpreted  as  an  a-discounted 
optimal  deterministic  stationary  policy,  and  that  the  corresponding  primal  optimal  solution 
yields  an  optimal  discounted  return. 

THEOREM  1.1:  There  is  one-to-one  correspondence  between  the  set  of  dual  basic  feasi- 
ble solutions  and  the  set  of  deterministic  stationary  policies. 

PROOF:  Let  (w,a)  be  any  dual  basic  feasible  solution,  that  is,  at  most  m of  the  w,a  's  are 
strictly  positive.  In  Lemma  2.2  we  showed  that  £ wia  > 0 for  each  /'€S.  Hence,  there  is  only 

a 

one  w,a  > 0 for  each  /€S:  Let  the  probability  of  making  action  a tA  when  the  process  is  in 
state  / € S be  defined  by 

(5.1)  d„  « -g-  = 1 or  0, 

b • 

depending  upon  whether  w,„  > 0 or  not.  Then  [d,a\  defines  a deterministic  stationary  policy 
denoted  by  a-  *. 

Using  the  argument  given  in  Theorem  2.1,  it  can  be  shown  that  the  deterministic  station- 
ary policy  <t  * corresponds  only  to  { v(0 ) . This  proves  the  theorem. 

From  this  theorem,  it  follows  that  the  dual  optimal  feasible  solution  obtained  by  using  the 
simplex  algorithm  may  be  interpreted  as  an  a-discounted  optimal  deterministic  stationary  pol- 
icy. The  corresponding  optimal  expected  discounted  return  is  obtained  from  the  primal  optimal 
feasible  solution. 

We  now  discuss  the  finite-state  and  action-set  Markovian  decision  models  and  the  associ- 
ated linear  programs  for  the  average-return  case.  As  in  the  discounted  case,  the  infinite  linear 
programs  given  in  Section  3 can  be  written  as  regular  linear  programs  given  by 


PRIMAL  IV 

DUAL  IV 

min  h 

max  r(i,  a) 

subject  to 

/ Q 

subject  to 

(«)  ” OyeS,  and 

h ~ vi  > r(i.a). 

tix»  - •’ 

/€S,  a€A. 

x,„  >0,  / €S,  a€A. 
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It  should  be  noted  that  the  dual  problem  has  (m  + 1)  constraints,  but  it  is  easy  to  show 
that  at  least  one  of  the  dual  constraints  is  a redundant  constraint.  Hence  any  basic  feasible 

solution  can  have  at  most  m of  the  x,a  different  from  zero.  Since  any  dual  feasible  solution  1 

{*,„}  must  satisfy  £x,0  > 0 for  each  itS,  we  conclude  that  all  basic  feasible  solutions  are  non- 

a 

degenerate.  It  can  be  shown  that  each  dual  basic  feasible  solution  corresponds  to  a determinis- 
tic stationary  policy  and  vice  versa.  Every  dual  basic  feasible  solution  can  be  interpreted  as  a , 

deterministic  stationary  policy  by  defining 

d,a  = i€S,  at  A. 

h 

If  the  simplex  method  is  used  to  solve  the  dual  problem,  we  will  always  get  an  optimal  i 

basic  feasible  solution.  This  optimal  solution  yields  an  average  optimal  deterministic  stationary 
policy,  and  the  corresponding  primal  optimal  solution  will  give  the  average  optimal  return 
h-  - «*»(/,  o-*),  itS. 

When  the  state  and  action  sets  are  finite,  the  above  methodology  will  give  an  alternate 
procedure  for  finding  the  optimal  policies  and  the  corresponding  returns.  The  methodology  also 
allows  us  to  conduct  a sensitivity  analysis  of  the  various  parameters  in  the  model,  using  existing 
linear-programming  codes. 

BIBLIOGRAPHY 


[1]  Blackwell,  D.,  "Discounted  Dynamic  Programming,  "Annals  of  Mathematical  Statistics, 

36,  226-235  (1965). 

[2]  Denardo,  E.  V.,  and  B.  L.  Fox,  "Multichain  Markov  Renewal  Programs,"  SIAM  Journal 

on  Applied  Mathematics,  16,  468-487  (1968). 

[3]  Denardo,  E.  V.,  "Markov  Renewal  Program  with  Small  Interest  Rates,"  Annals  of 

Mathematical  Statistics,  42,  477-496  (1971). 

[4]  Derman,  C.,  Finite  State  Markovian  Decision  Processes  (Academic  Press,  New  York,  1970). 
(51  Doshi,  B.,  "On  the  Continuous-time  Control  of  Queues,"  Technical  Report,  Department 

of  Statistics,  Rutgers  University,  New  Brunswick,  N.  J.  (1975). 

[6]  Duffin,  R.  J.,  and  L.  A.  Karlovitz,  "An  Infinite  Program  with  a Duality  Gap,"  Manage- 
ment Science,  12,  122-134  (1965). 

17]  Dynkin,  E.  B.  Markov  Processes  - 1 (Academic  Press,  New  York,  1965). 

18]  Evans,  J.  P.,  "Duality  in  Markov  Decision  Problems  with  Countable  Action  and  State 

Space,"  Management  Science,  15,  626-638  (1968). 

[91  Fox,  B.  L.,  "Finite-State  Approximations  to  Denumerable-State  Dynamic  Programming,” 
Journal  of  Mathematical  Analysis  and  Applications,  665-670  (1971). 

[10]  Gustafson,  S.  A.,  and  K.  0.  Kortanek,  "Numerical  Treatment  of  a Class  of  Semi-Infinite 

Programming  Problems,"  Naval  Research  Logistics  Quarterly,  20,  477-504  (1973). 

[11]  Howard,  R.  A.,  Dynamic  Programming  and  Markov  Processes  (The  M.I.T.  Press,  Cam- 

bridge, Mass.,  1960). 

[12]  Howard,  R.  A.,  "Research  in  Semi-Markovian  Decision  Structures,"  Journal  of  the  Opera- 

tions Research  Society  of  Japan,  6,  163-199  (1964). 

[13]  Kakumanu,  P.,  "Continuous  Time  Markov  Decision  Models  with  Applications  to  Optimi- 

zation Problems,"  Technical  Report  63,  Cornell  University,  Ithaca,  New  York  (1969). 

[14]  Kakumanu,  P.,  "Continuously  Discounted  Markov  Decision  Model  with  Countable  State 

and  Action  Space,"  Annals  of  Mathematical  Statistics,  42,  919-926  (1971). 


INFINITE  LINEAR  PROGRAMMING 


443 


115]  Kakumanu,  P.,  "Continuous  Time  Recurrent  Markovian  Decision  Processes  Average  Re- 
turn Criterion,"  Journal  of  Mathematical  Analysis  and  Applications,  52,  173-188 
(1975). 

[16]  Miller,  B.  L.,  and  A.  F.  Veinott,  Jr.,  "Discrete  Dynamic  Programming  with  a Small  Interest 

Rate,"  Annals  of  Mathematical  Statistics,  366-370  (1969). 

[17]  Mine,  H.,  and  Y.  Tabata,  "Linear  Programming  and  Continuous  Markovian  Decision 

Problems,"  Journal  of  Applied  Probability,  7,  657-666  (1970). 

[18]  Mine,  H.,  and  S.  Osaki,  Markovian  Decision  Processes,  (American  Elsevier  Publishing 

Company,  New  York,  1970). 

[19]  Osaki,  S.,  and  H.  Mine,  "Linear  Programming  Algorithm  for  Semi-Markovian  Decision 

Processes,"  Journal  of  Mathematical  Analysis  and  Applications,  22,  356-381  (1968). 

[20]  Ross,  S.  M.,  Applied  Probability  Models  with  Optimization  Applications,  (Holden-Day,  San 

Francisco,  CA,  1970). 

[21]  Simonnard,  M.,  Linear  Programming  (Prentice-Hall,  Englewood  Cliffs,  NJ,  1966). 

[22]  Veinott,  A.  F.,  Jr.,  "Discrete  Dynamic  Programming  with  Sensitive  Discount  Optimality 

Criteria,"  Annals  of  Mathematical  Statistics,  40,  1635-1660  (1969). 


RENEWAL  PROCESSES  OF  PHASE  TYPE 


Marcel  F.  Neuts 

University  of  Delaware 
Newark,  Delaware 

ABSTRACT 

This  paper  discusses  a class  of  analytically  and  numerically  tractable  renewal 
processes,  which  generalize  the  Poisson  process.  When  used  to  describe  in- 
terarrival or  service  times  in  queues,  these  renewal  processes  lead  to  computa- 
tionally explicit  solutions  which  involve  only  real  arithmetic.  Previous 
modifications  of  the  Poisson  process,  base  . on  the  Erlang  or  the  hyperexponen- 
tial distributions,  appear  as  particular  cases. 


1.  INTRODUCTION 

In  many  stochastic  models,  tractable  analytic  or  numerical  results  are  usually  only 
obtained  if  certain  random  variables  are  assumed  to  have  a negative  exponential  distribution. 
This  accounts  for  the  wide  use  of  the  Poisson  process  as  an  arrival  process  in  the  analysis  of 
queues  and  counters,  the  birth-and-death  assumptions  underlying  epidemic  models,  and  the 
negative  exponential  durations  ascribed  to  service  times,  lead  times,  headway  in  traffic,  and  a 
large  variety  of  other  random  time  intervals.  The  classical  memory-less  property,  which  elim- 
inates the  drastic  growth  in  dimensionality  due  to  conditioning,  is  the  underying  source  of  all 
simplifications  that  we  owe  to  the  negative  exponential  distribution. 

The  limitations  of  the  exponential  distribution  in  modeling  real  durations  are  well-known. 
A large  probability  is  assigned  to  the  shorter  time  intervals,  and  the  proper  unimodality  or  mul- 
timodality of  many  real  situations  cannot  be  represented.  This  was  recognized  by  A.  K.  Erlang. 
It  led  him  to  introduce  the  probability  distributions  which  bear  his  name.  In  practice,  it  is  now 
common  to  assume  Erlang  or  hyperexponential  (finite  mixtures  of  negative  exponentials)  distri- 
butions to  model  random  time  durations  which  are  too  far  removed  from  the  exponential  case. 
These  distributions  have  a greater  versatility  and  allow  for  relatively  tractable  expressions  under 
repeated  conditioning. 

With  these  desirable  properties  in  mind,  the  author  (3]  introduced  the  probability  distribu- 
tions of  phase  type , of  which  the  Erlang  and  hyperexponential  distributions  are  very  special  cases. 
Discusssions  of  the  algorithmic  simplifications  introduced  in  the  study  of  some  problems  in 
queues  and  branching  processes  are  given  in  Refs.  [4],  (5),  16]  and  [7], 

The  present  paper  deals  specifically  with  renewal  processes  in  which  the  distribution  of  the 
time  between  renewals  is  of  phase  type.  The  material  developed  here  is  basic  in  the  discussion 

•This  research  was  sponsored  by  the  Air  Force  Office  of  Scientific  Research,  Air  Force  Systems  Command,  USAF, 
under  Grant  No.  AFOSR-72-2350  B The  United  States  Government  is  authorized  to  reproduce  and  distribute  re- 
prints for  governmental  purposes  notwithstanding  any  copy  right  notation  hereon. 
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of  certain  queueing  models  which,  to  date,  have  not  been  solved  in  a computationally  con- 
venient form  [7j. 

HISTORICAL  BACKGROUND 

We  first  review  some  definitions  and  state  some  results  established  earlier.  In  this  paper 
we  only  concern  ourselves  with  probability  distributions  of  phase  type  on  [0,  °°),  bearing  in 
mind  that  there  is  an  entirely  analogous  development  of  probability  distributions  of  phase  type 
on  the  nonnegative  integers. 

We  consider  an  (m+  l)-state,  continuous-parameter  Markov  chain  with  states  1 

m+1,  whose  infinitesimal  generator  @has  the  form 

(1)  Q = O ol* 

where  7”  is  a nonsingular  mxm  matrix  and  T°  is  an  m- vector.  The  diagonal  elements  of  T are 
negative.  All  other  entries  of  T and  the  components  of  T°  are  nonnegative.  Moreover 

(2)  7e  + T°-  O where  e - (1,  1 1)'. 


The  state  m- 1- 1 is  absorbing.  We  require  that  all  other  states  be  transient.  The  necessary 
and  sufficient  condition  for  this  is  that  the  inverse  T~ 1 exists.  In  this  case  eventual  absorption 
into  the  state  m+1  from  any  initial  state  /€  (1 . ....  m}  is  certain. 

The  vector  of  initial  probabilities  is  denoted  by  (o,arm+1),  where  a is  an  m- vector  such 
that  0 < ae  < 1.  The  probability  distribution  F (•)  of  the  time  till  absorption  in  the  Markov 
chain  Q is  then  easily  seen  to  be 

(3)  F(x)  - 1 - aexp(fx)  e,  for  x > 0. 


The  probability  distribution  F(.)  is  said  to  be  of  phase  type.  Henceforth  this  phrase  will  be 
rendered  as  'F(-)  is  PH."  The  pair  (o.  T)  is  called  a representation  of  F(  ).  If  > 0,  the 
distribution  F(.  ) has  a jump  of  height  am+,  at  the  origin.  All  other  probability  mass  is  distri- 
buted on  (0,  <»),  according  to  a density  given  by 

(4)  <t>(u)  -or  exp(7V/)  T°,  for  u > 0. 

The  moments  pk,  k ^ 1,  about  the  origin  all  exist  and  are  given  by  the  formula 

(5)  -(-!)* /r!«r-*e,  for*  > 1. 


EXAMPLES,  (a)  For  the  exponential  distribution  with  parameter  A,  the  matrix  Q is  given 
by 

-X  A 

(6)  Q “ 0 0'  and  o,-l.  a2-0. 

so  that  F(  ) then  has  the  simple  representation  (1,  -X). 

(b)  The  generalized  Erlang  distribution  obtained  by  tiie  convolution  of  m exponential  dis- 
tiibutions  with  parameters  X, Xm  has  as  one  of  its  representations  the  pair  (o.  T)  given 
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(7) 


a - (l,  0 0) 


0 

0 


6 


with  T°  — (0 0,  Am)'. 

(c)  The  hyperexponential  distribution 

(8)  Fix)  - 2>,(1  - e ' ).  x >0, 

i- 1 

may  be  represented  by  a-(a( am)  and  T = diagl-K^  -A2,  ....  -Am),  with 

T°-  (A, A 

For  any  representation  (a.  T),  we  can  define  a matrix  Q*  by 

(9)  Q*  — T + T°A 

where  T is  an  mxm  matrix  with  m identical  column  vectors  given  by  the  vector  T°.  The 

matrix  A°  is  defined  to  be  (1  - am+|)~l  diagla am).  The  matrix  @*is  easily  seen  to  be 

a conservative  stable  matrix.  It  may  be  considered  to  be  the  transition  probability  matrix  of  an 
m-state  Markov  chain,  which  has  a close  relationship  to  the  probability  distribution  F(-).  As 
shown  in  [31,  we  may  always  delete  possibly  superfluous  states  in  the  original  chain  to  insure, 
without  loss  of  generality,  that  the  matrix  Q*  is  irreducible. 

The  significance  of  0*is  as  follows.  At  any  time  that  an  absorption  occurs  in  the  Markov 
chain  Q,  we  instantaneously  perform  one  or  more  independent  multinomial  trials  with  probabili- 
ties a, am+i.  until  one  of  the  outcomes  1 m occurs.  The  state  / € ( 1 , ....  n)  so 

obtained  is  then  treated  as  a new  initial  state  for  the  Markov  chain  Q.  The  process  so  obtained 
is  an  (m-l- 1) -state  Markov  chain  in  which  the  state  m+ 1 is  an  instantaneous  state.  The  process 
obtained  by  requiring  the  path  functions  to  be  right-hand  continuous  is  again  a Markov  chain 

with  the  m states  {1,  2 m)  and  the  infinitesimal  generator  Q*.  It  is  routinely  verified  that 

the  times  between  the  (instantaneous)  visits  to  the  state  nt+ 1 are  independent  and  identically 
distributed  with  the  common  distribution  F(  ). 

The  remainder  of  this  paper  deals  with  a unified  treatment  of  renewal  processes  of  phase 
type.  For  use  in  the  sequel,  we  recall  the  following  result,  proved  in  Ref.  [31 


THEOREM  1:  If  the  probability  distribution  F(  ) is  PH  with  representation  (a,  T)  then 
the  probability  distribution 

(10)  /••(*)  - -V  fA  [1  - F(u)  J du.  x >0. 

Mi  Jo 

is  PH  with  the  representation  In.  T ) where  n is  the  unique  probability  vector,  satisfying 

nQ'  - w(r  + T°A°>  -0, 

(11)  we  - 1. 
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REMARK:  The  straightforward  proof  of  Theorem  1 is  given  in  Ref.  [3).  It  is  worth 
pointing  out  that  this  result  is  highly  intuitive.  In  the  stationary  version  of  the  Markov  chain 
Q *,  ft , is  the  probability  of  being  in  state  /,  1 < / < m at  a given  time.  The  PH-distribution 
with  representation  (»r.  T)  is  clearly  the  probability  distribution  of  the  time  till  the  next  visit  to 
the  instantaneous  state  m+1,  i.e.,  until  the  next  renewal. 

COROLLARY  1: 

(12)  (1  1/m',. 

PROOF:  Formula  (1 1)  leads  to 


(13)  *T-  -(1  -am+](irTa)a. 
and  hence 

(14)  ire  - -0  — am+1) («n  a T~'  e - (1  -am+1)-'  (bTVi  - 1. 

THEOREM  2:  The  renewal  density  of  an  ordinary  PH-renewal  process  is  given  by 

(15)  0(f)  = (1  - am+1)-2aexpl(F  + T°A  °)t]  T°,  for  f > 0, 
and  we  have 

(16)  lim  0(f)  - 4-. 

A*  i 

PROOF:  The  quantity  d>(t)dt  is  the  expected  number  of  renewals  in  It,  t + dt).  If  at 
least  one  renewal  occurs,  the  expected  number  of  renewals  in  [f,  f + dt)  is  given  by 
(1  - am+|)  The  probability  that  there  is  at  least  one  renewal  in  [/,  f + dt)  is  also  the  proba- 
bility that,  in  the  Markov  chain  Q *,  a visit  to  the  instantaneous  state  m+ 1 occurs  during  that 
interval  of  time.  The  latter  probability  is  given  by 

(17)  (1  -am+1)-'aexp«?*r)  T°dt, 
so  that  (15)  follows. 

Since  Q*  is  irreducible,  so  is  the  stochastic  matrix  exp(£)*f).  In  fact,  one  may  prove  that 
exp(0*f)  is  strictly  positive,  and  therefore  aperiodic.  The  limit  matrix  of  exp(Q*f),  as  f -*  °», 
is  given  by  II,  with  n/;  — ir ,,  for  1 < /,  j < m.  It  follows  that 

(18)  lim  0(f)  - (1  -«J-!«nT"  - (1  (#T^  - “r . 

/—  ft , 

by  (12). 

Remarks  on  Computation 

We  see  that  0(f)  - (1  - am+|)"'  0t(f),  where  0^'J  is  the  renewal  density  of  a PH 
renewal  process  with  underlying  PH-distribution  F ((-)  with  representation  (fi,  T),  where 
0-(  1 - am+1) _l  a.  We  can  therefore  conveniently  assume  that  a „+,  - 0,  in  computing  <f>(/). 
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We  only  need  to  solve  the  system  of  linear  differential  equations 
v'(r)  - v(r)  (T  + T°A°),  for  t > 0. 

(19)  v(0)  — a, 

by  any  one  of  a large  number  of  numerical  methods.  For  each  computed  vector  v(r),  </>(r)  is 
then  given  by 

(20)  <M/)-v(/)T° 


PH-renewal  densities  can  therefore  be  readily  computed  by  entirely  elementary  methods. 
This  is  of  some  practical  and  pedagogical  interest.  We  also  see  that  the  stationary  renewal  den- 
sity <£j(r)  is  obtained  by  choosing  the  initial  state  of  the  chain  Q*  according  to  the  probability 
vector  ir.  We  obtain  the  expected  result 

02(r)  = (1  - c*m+1)  _l  it  exp(Q't)  T ° 

(21)  - (1  -am+|)-'«-T°=  1/m,. 

2.  THE  NUMBER  OF  RENEWALS  IN  AN  INTERVAL 

Let  P,j(n,  r)  be  the  conditional  probability  that  at  time  t the  Markov  chain  Q*  is  in  the 
state  7 € { 1 , ...  , m ) and  that  n renewals  have  occurred  in  the  interval  (0,  r),  given  that  at  time 
t — 0+  the  chain  Q*  was  in  the  state  / € { 1 , ...,  m ).  The  matrix  with  entries  P„in,  t ) will  be 
denoted  by  P(n,  t).  It  is  then  easy  to  see  that  the  matrices  P(n,  t ) satisfy  the  recurrence  rela- 
tions 


Pi 0,  r)  - exp(7V),  for  t ^ 0, 
and 

(22)  Pin,  t)  = (1  - am+|)  , Jo'exp[7-(r  - «)]  T°A°Pin  - v.  u)  du 

- (1  - om+1)  Jo  Pin  - V,  u)T°A  °exp[r(f  - L')l  du, 

*>  — 1 

for  n ^ 1,  r > 0. 

The  matrix  probability  generating  function 

(23)  P'iz,  t)  - £ Pin.  t)z\  |z|  < 1. 

n-0 

is  given  by  the  following  theorem: 

THEOREM  2: 

(24)  P*(z,  t)  - expjlf  + (1  - am+|  z)~'  (1  - am+1)  z 7">^8)rl,  for  t > 0. 


PROOF:  The  recurrence  relations  (22)  lead  to 
P*iz.  i ) - exp iTi)  + 

(1  - am+|Z)-1  (1  - am+|)z  Jo'exp(n/  - «))  T°A  Viz.  u)  du. 


(25) 
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It  is  well-known  that  exp(-7>)  exists  and  is  the  inverse  of  exp(7>)  (1).  Left-multiplying  in 

(25)  by  exp(—  77)  and  differentiating  the  resulting  expression  with  respect  to  /,  we  obtain, 

exp (-77)  ~ i ) - exp(— 7>)  T P*iz,  t) 

at 

(26)  + (1  - am+1r)-'  z(l  - a m+i)  exp(-7V)  T°A  °/>*(z,  t). 
which  leads  us  to  the  differential  equation 

(27)  -f-  P'iz.  t)  - [r+  (1  — am+, z)_l  ( 1 — am+| ) zT'A']  P'iz,  t) . 
ot 

with  the  initial  condition  P*iz,  0)  - /.  It  now  follows  directly  that  P*iz,  t)  is  given  by  (24). 

REMARKS:  (a)  Formula  (24)  generalizes  the  classical  formula  for  the  probability  gen- 
erating function  of  the  Poisson  distribution.  For  m— 1,  T— X,  T°=X,  and  a2“0,  a,=  l,  we 
obtain 

(28)  P'iz,  t ) - exp(— X + Xz)  t. 

(b)  By  a series  expansion  with  respect  to  z in  (27),  or  by  repeating  the  argument  used  in 
proving  (26),  we  see  that  the  matrices  Pin,  t ) satisfy  the  system  of  linear  differential  equa- 
tions. 


Pi 0,  t)  = T PiO,  t ) - pi 0,  t)  T, 
and 

P (n,  t)~T  Pin,  t)  + (1  - am+1)  £<+',  T°A°P(n  - v,  t) 

I/-1  • 

(29)  - Pin,  t)  T + (1  - am+|)  £«;+*,  Pin  - t)T°A  °. 

*-i 

for  n ^ 1,  t > 0,  with  pin,  0)  — 8 „„/,  for  n > 0. 

Except  for  very  special  cases,  it  will  be  necessary  to  solve  the  system  (29)  by  numerical 
techniques.  We  also  note  that  in  the  case  am+1  — 0,  which  occurs  in  most  practical  applica- 
tions, all  the  preceding  expressions  simplify  considerably. 

THEOREM  3:  For  r > 0,  we  have 


(30) 


-2-  P*iz,  t)  e 
dt 


r-1 


t e + (1  - am+1)-'  [/  - exp(0*f)l  (tTI  - Q*)-1  T°, 


where  r*  > maxi— 

PROOF:  Expanding  the  matrix  exponential  in  Formula  (24)  and  differentiating  with 
respect  to  z,  we  obtain 


(31) 


-f 

dt 


- (1  - «m+1)->  i~"i  iQ •)'  T'A'iQ*)'--' 


j-i 


t ^ ' *>—o 
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Right-multiplying  on  the  right  by  e we  obtain,  since  Q't  — O, 

(32)  t)t  -(l-aj-'i^^'11, 

3*  Jr-1  »-l  n' 

- (1 J0'exp(OM</«T°. 

The  evaluation  of  the  latter  integral  is  of  some  independent  interest,  since  the  (/,  y)-entry  of 
the  matrix  J*  exp (Q*u)du  is  the  expected  amount  of  time  spent  in  the  state  j in  (0,  r),  given 
that  the  Q'- chain  starts  in  the  state  /. 

It  is  easily  verified  that  the  matrix 

(33)  P\-  I + Q*. 

T 

is  irreducible,  is  stochastic,  and  has  the  invariant  probability  vector  it.  This  implies  that  the 
matrix  / — /’,  + n is  nonsingular  [2J.  We  have  the  relations 

(34)  / - /»,  + n - n - n-nn-^r  <?•. 

J T * 


We  now  see  that 


j °°  •I'+i  j 

0 exP(G*“)  du  n - - 0*  - I «?•)'  n - -L  q- 


- nr  - (exp(0*r)  - /]. 

T * 


from  which  it  follows  that 


4-  ')«  -(l-oj-'tnr 

\dz  Jr— 1 


+ (1  - «„+,)-'  [/  - exp(0*r)l  n - 


Using  Corollary  1,  we  obtain  (30). 


COROLLARY  2:  The  expected  number  H(t)  of  renewals  in  (0,  r]  is  given  by 
HU)  • (1  — am+1)_laf-~  P'(z,  r)el 


(37)  -MI  t + (1  -am+l)-J«[/-exp«?*f)l  (r*n-  (?*)-'  T°. 

and  the  expected  number  of  renewals  in  (0,  r]  is  given  by  H(t)  + (1  - 

REMARK:  Except  for  the  inverse  of  the  matrix  r *n  - Q*,  which  needs  to  be  evaluated 
only  once,  the  numerical  computation  of  the  renewal  function  again  reduces  to  the  computation 
of  a exp(Q*r),  for  various  values  of  r.  This  may  be  done  by  a routine  numerical  solution  of  a 
well-behaved  system  of  linear  differential  equations  with  constant  coefficients. 


452 


M.  F.  NEUTS 


3.  NUMERICAL  METHODS 


In  a number  of  applications  of  PH-distributions,  it  is  necessary  to  evaluate  the  matrices 
Pin,  t)  for  0 ^ n < N and  0 ^ t < T*.  Usually  7"* is  given  and  N needs  to  be  determined  so 
that  the  components  of  the  vector 

(38)  £ Pip.  T*)  e 

K-/V  + 1 

are  all  less  than  a preassigned  e. 


If  N and  T*  are  known,  the  system  of  differential  equations 
(39)  Pi 0,  f)  = T P(0,  t), 

and 

P'in,  t)  = T Pin,  t)  + £<+',  T°A°  Pin  - v,  t),  for  1 < n < N. 

v-\ 

with  initial  conditions  P( 0,  0)  = /,  Pin,  0)  = 0,  for  1 < n < jV,  can  in  most  cases  of  practical 
interest  be  solved  by  a classical  procedure  such  as  the  Runge-Kutta  method.  Although  the 
coefficient  matrix  of  the  system  (39)  has  a very  simple  structure,  shown  in  (40),  the  order  in 
the  system  is  usually  sufficiently  high  that  its  solution  by  a "theoretical"  method  such  as  the 
spectral  method  is  impractical  and  lacking  in  numerical  accuracy. 


The  coefficient  matrix  of  the  system  (39)  is  given  by 
T «m+i  T°A  ° «m+ 1 T°A°  • 


(40) 


C(  A) 


*m  + l T°A 

T 

0 


0 

0 


<*m+\T°A 
am  + \T°A  ' 
am+\T°A  ' 
a£;iT°A  - 


»,*i  T°A 
T 


and  C(N)  is  a matrix  of  order  Nm.  In  view  of  the  special  structure  of  C(N),  it  is  not  neces- 
sary to  store  this  large  matrix  in  implementing  a numerical  integration  method  such  as  the 
Runge-Kutta  procedure. 


The  matrix  T is  also  frequently  sparse,  and  this  can  be  used  in  practice  to  obtain  substan- 
tial savings  in  the  number  of  arithmetic  operations  involved  in  solving  the  system  (39). 

In  most  practical  cases,  the  system  (39)  is  also  not  stiff.  Any  problems  due  to  stiffness  are 
usually  apparent  in  the  entries  of  the  matrix  T.  If  one  attempted  to  approximate,  e.g.,  the 
degenerate  distribution  at  c > 0,  by  an  Erlang  distribution  of  order  m and  parameter  X - c/m, 
one  would  obtain  not  only  a very  high  value  of  m,  but  also  a stiff  system  of  differential  equa- 
tions. Techniques  based  on  distributions  of  phase  type  are  clearly  not  suitable  in  this  case. 
Theoretical  results  on  these  problems  provide  only  broad  guidelines,  and  a certain  amount  of 
numerical  experimentation  and  the  use  of  build-in  accuracy  checks  appears  to  be  required  in 
practice. 


The  computer  storage  requirements  depend  clearly  on  the  sizes  of  m and  N.  In  some 
applications,  we  are  not  interested  in  the  matrices  Pin,  i)  themselves,  but  in  some  quantity 
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expressed  in  terms  of  them.  As  discussed  in  Ref.  [7],  the  applications  in  queueing  theory 
involve  the  sequence  of  matrices  A„,  n ^ 0,  defined  by 

(41)  A,- Pin.  t)  d KU). 

where  K(-)  is  a probability  distribution  on  [0,  °°).  The  evaluation  of  an  adequate  number  of 
terms  of  the  sequence  { A„ ) requires  first  a truncation  of  the  integral  in  (41),  followed  by  a 
numerical  integration  using  the  computed  matrices  P(n,  I ) at  a sufficient  number  of  /-points. 
We  can  avoid  storing  the  matrices  Pin,  t ) for  a large  number  of  / points  by  evaluating  the 
integrals  in  (41)  by  a "progressive"  integration  procedure  such  as  Simpson’s  rule.  For  special 
distributions  Kit),  there  may  be  a substantial  gain  in  computational  effort  by  using  an  appropri- 
ate quadrature  method,  say  Laguerre  quadrature,  if  K(-)  is  a gamma  distribution.  These  are, 
however,  routine  matters  of  numerical  analysis,  whose  discussion  here  would  only  repeat  classi- 
cal material. 


We  conclude  by  listing  a few  practical  problems,  which  can  be  handled  by  ad  hoc  general 
procedures,  but  for  which  a more  refined  analysis  would  be  welcome. 


For  a given  T*  > 0,  we  need  to  determine  N,  so  that  the  conditons  stated  in  (38)  holds. 
More  tractably,  we  can  determine  N so  that  the  probability  of  having  more  than  N renewals  in 
[0,  T*[  is  small.  As  a crude  but  useful  procedure,  we  can  use  known  approximations  to  the 
mean  and  the  standard  deviation  aiT*)  of  the  number  of  renewals  in  (0,  7'"],  and  choose,  e.g., 
N to  be  the  smallest  integer  to  execeed  HiT*)  + 3<r (T*).  For  larger  values  of  T*,  the  asymp- 
totic normality  of  the  number  of  renewals  in  [0,  Tl  can  be  used  for  the  same  purpose.  It 
would  be  of  interest  to  have  a refined  analysis  of  the  remainder  vector,  given  in  (38),  but  this 
does  not  appear  easy. 


We  also  note  that  the  sequence  |(1  - am  + ]) _l  a Pik,  t ) e,  k ^ 0)  defines  a discrete  pro- 
bability density,  with  parameters  a,  T,  and  /,  which  generalizes  the  Poisson  density  with  param- 
eter X/.  There  are  a variety  of  particular  cases,  involving  only  a small  number  of  parameters, 
which  may  be  of  interest  as  counting  distributions  derived  from' modifications  of  the  Poisson 
process.  If  we  consider  the  particular  case  given  in  (7),  we  obtain  the  probability  density 


(42) 


p*  - a Pik,  t)  e 


m- 1 


j'-O 


imk  + k)!  ’ 


for  k ^ 0. 


Few  particular  cases  are  as  tractable  analytically  as  this  case.  Even  for  the  hyperexponential 
case,  the  explicit  form  of  the  probabilities  pk  is  forbiddingly  complicated. 
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ABSTRACT 

We  present  techniques  Tor  classifying  Markov  chains  with  a continuous  state 
space  as  either  ergodic  or  recurrent.  These  methods  are  analogous  to  those  of 
Foster  for  countable  space  chains.  The  theory  is  presented  in  the  first  half  of 
the  paper,  while  the  second  half  consists  of  examples  illustrating  these  tech- 
niques. The  technique  for  proving  ergodicity  involves,  in  practice,  three  steps: 
showing  that  the  chain  is  irreducible  in  a suitable  sense;  verifying  that  the  mean 
hitting  times  on  certain  (usually  bounded)  sets  are  bounded,  by  using  a "mean 
drift*  criterion  analogous  to  that  of  Foster;  and  finally,  checking  that  the  chain 
is  such  that  bounded  mean  hitting  times  for  these  sets  does  actually  imply  ergo- 
dicity. 

The  examples  comprise  a number  of  known  and  new  results:  using  our 
techniques  we  investigate  random  walks,  queues  with  waiting-time-dependent 
service  times,  dams  with  general  and  random-release  rules,  the  s-S  inventory 
model,  and  feedback  models.  - 


1.  INTRODUCTION 

Since  the  introduction  of  the  embedded  chain  method  by  Kendall  [9],  Markov  chain 
analysis  has  been  recognized  as  an  important  tool  in  many  branches  of  operations  research. 
The  most  closely  studied  chains  have  undoubtedly  been  those  which  are  integer-valued.  This  is 
so  partly  because  a very  complete  theory  exists  for  analyzing  such  chains,  but  a contributing 
factor  has  also  been  the  formulation  (since  the  results  of  Foster  [6])  of  readily  verifiable  criteria 
for  classifying  integer-valued  chains  as  ergodic  or  as  recurrent.  Recent  developments  in 
Markov  chain  theory  now  enable  these  criteria  to  be  extended  to  cover  the  classification  of 

‘Currently  at  Department  of  Statistics,  Yale  University,  New  Haven,  Connecticut. 
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chains  taking  values  in  more  general  spaces.  Tweedie  [19]  has  given  a review  of  this  general 
theory. 

In  this  paper  the  special  case  of  Markov  chains  whose  state  space  is  the  real  line,  or  a 
closed  subset  of  the  real  line,  will  be  considered.  We  shall  develop  techniques  for  proving  the 
existence  of  unique  stationary  distributions,  and  for  showing  that  the  n-step  transition  probabili- 
ties converge  to  this  stationary  distribution.  These  techniques  enable  us  to  classify  a number  of 
the  continuous-valued  chains  which  occur  naturally  in  operations  research;  applications  to  ran- 
dom walks,  waiting  times  for  queues,  capacities  of  dams,  inventory  problems,  and  feedback 
chains  are  given. 

2.  CONTINUOUS- VALUED  MARKOV  CHAINS 

In  order  to  motivate  the  description  of  continuous-valued  chains,  we  begin  with  a review 
of  the  corresponding  more  familiar  concepts  for  the  integer-valued  case. 

The  evolution  of  a (time  homogeneous)  integer-valued  chain  is  described  by  its  transition 
matrix  P = [P(i,  j)]  defined  by 

Pd.  j ) - PrU„  = j\X„-\  = /•)'. 

Suppose  that  the  chain  is  irreducible , i.e.,  for  every  pair  (/,  j)  there  exists  an  n such  that 

Pr{X„=j\X0-  /)  > 0. 

In  the  analysis  of  such  chains,  a question  of  fundamental  interest  concerns  the  existence  of  a 
unique  stationary  distribution  (ir(fc)]:  that  is,  a distribution  satisfying 

(2.1)  n(k)  = 2>(/)  PU.  k) 

j 

for  all  k.  If  such  a distribution  exists  {A'„J  is  said  to  be  ergodic  (or  positive  recurrent). 

A concept  weaker  than  ergodicity  is  recurrence.  Consider  the  hitting  times 

Tk  - inf{»  > 0:X„  - k). 

The  chain  (JTn)  is  said  to  be  recurrent  if  these  variables  are  proper  for  any  starting  point  X& 
that  is,  if 

Pr[Tk  < o°|2r0-y}  - 1 

for  all  j and  k.  It  is  well  known  (Feller,  Ref.  [7],  Chapter  XV)  that  for  an  irreducible  chain 
recurrence  is  equivalent  to  the  single  condition 

Pr{T0  < »|*o-0]  - 1. 

Further,  the  chain  is  ergodic  if  and  only  if 

(2.2)  £'[r0|A'o-0]  < oo 
and  the  stationary  distribution  is  then  given  by 

n(k)  - (£[r*|*o-  *])”'  > 0 

for  each  k.  In  all  but  the  simplest  cases  it  is  impossible  to  solve  (2.1)  directly,  or  to  find  the 
distribution  of  T0  conditional  on  X0  — 0 explicitly  enough  to  check  for  recurrence  or  ergodicity. 
However  Foster  [6],  and  later  Mauldon  [12]  and  Pakes  [14],  have  given  sufficient  conditions  for 
recurrence  and  ergodicity  of  irreducible  integer-valued  chains.  We  extend  these  conditions  to 
continuous-valued  chains. 
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Consider  a time-homogeneous  Markov  chain  [X„],  with  state  space  X which  we  will  usu- 
ally assume  to  be  a closed  (but  not  necessarily  bounded)  subset  of  the  real  line  (— °°,  »).  (All 

our  results  can  be  given  virtually  unaltered  when  1 is  taken  as  a closed  subset  of  a higher 

dimensional  Euclidean  space,  but  for  ease  of  exposition  we  generally  restrict  ourselves  to  the 
one  dimensional  case  here.)  The  evolution  of  the  chain  is  described  in  one  dimension  by  the 
collection  of  distribution  functions 

Fx{y)  = Pr(X„+l  < y \X„  - *}; 

however,  we  will  find  it  easier  to  work  with  the  corresponding  measures 

P(x,  A)  = Pr[X„  + 1 6 A \X„  - x) 

induced  in  one  dimension  by  the  distributions  Fx\  that  is,  if  A = (a.  b],  then  P(x,  A)  — 

Fx(b)  - Fx(a),  and  P(x,  •)  is  then  extended  to  all  the  Borel  subsets  of  the  real  line  in  the 

usual  way.  We  assume,  in  order  that  the  chain  be  well-defined,  that  for  each  A € ‘J  (the  a- 
field  of  Borel  subsets  of  30  the  function  P(-.  A)  is  measurable,  and  for  each  x,  P(x,  ■)  is  a 
probability  measure  on  5F. 


As  in  the  discrete  case,  to  classify  such  a chain  we  need  a notion  of  irreducibility. 

DEFINITION:  We  say  {Xn}  is  <t>- irreducible  if  there  exists  a nonzero  measure  <t>  on  Fsuch 
that,  for  any  x € "X  and  A 6 CJ  with  <b(A ) > 0,  there  is  an  n for  which  P"(x.  A ) > 0. 

When  3(  is  countable,  taking  <t>  as  counting  measure  leads  us  back  to  the  usual  concept  of 
irreducibility.  For  general  X <t>  need  not  have  any  atoms  (<t>  will  often  be,  for  example,  Lebes- 
gue  measure)  but  if  it  does  have  atoms  then  the  analysis  is  greatly  simplified  (see  Section  10). 


We  shall  call  a ^-irreducible  chain  [Xn]  ergodic  if  it  has  a unique  stationary  distribution, 
i.e.  a probability  measure  7r  on  ? satisfying,  for  every  A € X 

(2.3)  n(A)  “ J*(  n(dx)  P(x,  A). 


If  (X,}  is  ^-irreducible  then  (Tweedie,  Ref.  [19],  Section  4)  there  is  at  most  one  stationary  dis- 
tribution; if  further  [X„]  is  ergodic  then  the  n-step  transition  probabilities  converge  to  the  sta- 
tionary distribution  n in  the  strong  Cesaro  sense  that 


(2.4) 


sup 

ACF\ 


X P"'(x.  A)  — 7T( A) 

m- 1 


0 


for  7r-almost  all  x. 


Thus  ergodicity  has  connotations  for  continuous  state  spaces  similar  to  those  for  the 
discrete  case.  Moreover,  there  is  again  a close  relationship  between  ergodicity  and  the  finite- 
ness of  the  means  of  the  hitting  times 

Ta  - inf (w  > 0:  X„  € A). 

This  connection  is  not  as  simple  as  the  necessary  and  sufficient  condition  given  by  (2.2)  for 
discrete  chains.  But  it  can  be  shown  that,  given  certain  conditions  on  the  chain,  ergodicity  is  a 
consequence  of: 

(2.5)  sup  E[Ta\X0  - x]  < 

x(.A 

provided  A is  one  of  a certain  class  of  sets  determined  by  the  preliminary  conditions  satisfied  by 
the  chain.  One  of  the  main  aims  of  this  paper  is  to  detail  the  conditions  which  lead  to  a useful 
class  of  sets  in  this  context.  For  example,  we  shall  show  that  ^-irreducibility  plus  certain  con- 
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tinuity  constraints  on  the  transition  probabilities  implies  that  we  need  only  verify  (2.5)  for  a 
single  bounded  set  A of  positive  </>-measure,  in  order  to  prove  that  the  chain  is  ergodic.  In  gen- 
eral, we  call  any  set  A such  that  (2.5)  is  a sufficient  condition  for  ergodicity  of  (*„)  a test  set 
(for  ergodicity ) for  that  chain. 

In  the  next  four  sections  we  describe  some  suitable  test  sets  for  four  important  classes  of 
Markov  chains  which  occur  in  operations-research  models.  For  such  chains  then,  ergodicity  can 
be  proved  by  finding  a test  set  satisfying  (2.5).  This  will  always  be  achieved  by  applying  the 
following  result. 

THEOREM  2.1  (Tweedie,  Ref.  [19]),  Theorem  4.1):  Let  g be  a nonnegative  measurable 
function  on  %.  If  for  some  « > 0 and  A € J, 

(2.6)  J"  P(x,  dy)  g(y)  < g(x)  - e for  xU', 
i.e.,£[tf(*|)|*o  — x]  ^ g(x)  - e for  x € Ac, 

then  we  have  the  bounds 

(2.7)  E[Ta |*0-x]  ^ g(x)/e  for  x € Ac, 
and 

(2.8)  E[Ta\X0=  x]  < 1 + J*  P(x,  dy)  g(y)/e  for  x € A. 

A very  common  choice  for  the  test  function  g is  g(x)  - x when  the  space  3C  = (0,  °°).  This 
will  be  the  case  for  the  examples  of  Sections  10,  11,  12,  and  14.  Thus  it  seems  worthwhile  to 
show  explicitly  how  Theorem  2.1  can  be  used  for  that  choice  of  g. 

THEOREM  2.2:  Suppose  X = [0.  °°)  and  that  there  exist  « > 0,  M < °°  and  a 
bounded  A € 5 such  that 

(2.9)  £[*,1*0  - x]  < x - € for  x € Ac 
and 

(2.10)  £[*,!*<> -x]  < M for  x € 4. 

Then 

sup  £[ 7"^ | *0  - x]  < °°. 

*€  A 


PROOF:  Apply  Theorem  2.1  with  *(x)  = x,  noticing  that  x/«  is  bounded  on  A and  that 
the  right-hand  side  of  (2.8)  is  bounded  by  1 + M/t. 

In  most  of  the  cases  which  we  shall  consider  the  space  X is  [0,  °°)  and  the  test  set  will  be 
a bounded  interval  [0,  /3],  We  can  then  interpret  (2.9)  as  "mean  drift  towards"  this  test  set. 
With  this  sort  of  interpretation,  many  of  our  results  are  more  intuitively  meaningful:  they  can 
be  seen  as  providing  some  delineation  of  the  class  of  chains  for  which  mean  drift  towards  "rea- 
sonable" sets  does  in  fact  imply  ergodicity. 

The  discussion  above,  and  the  method  we  advocate,  can  be  summarized  in  the  following: 

ERGODICITY  TECHNIQUE:  To  prove  ergodicity  for  a given  Markov  chain  (*„),  carry 
out  the  following  three  steps: 


STEP  I.  Identify  a suitable  <t>  and  show  (*„|  is  (/,-irreducible  for  this  </>. 
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STEP  II.  Identify  possible  test  sets  for  the  chain. 

STEP  III.  Apply  Theorem  2.1  (or  Theorem  2.2)  to  one  of  these  test  sets  to  prove  bound- 
edness of  the  mean  hitting  times  as  specified  by  (2.5). 

The  reader  who  is  primarily  interested  in  applications  could  now  turn  straight  to  Section  9, 
which  is  the  beginning  of  the  examples  segment  of  this  paper.  Of  course  some  back- 
referencing  would  then  be  required:  to  Sections  3-6  to  see  how  the  test  sets  have  been 
identified;  to  Sections  7 for  ^-irreducibility;  and  possibly  to  Section  8 as  well,  where  we 
describe  a concept  of  recurrence  which  generalizes  the  one  for  discrete  chains. 

3.  TEST  SETS  WHEN  <t>  HAS  AN  ATOM 

Sometimes  <t>  can  be  chosen  to  have  an  atom  at  some  point  a,  i.e.  (a)  can  be  reached 
from  every  point  in  the  state  space  (see  Sections  10  and  11). 

THEOREM  3.1  (Tweedie,  Ref.  [19],  Section  5):  If  <t>  has  an  atom  at  a then  (a)  is  a test 
set  for  any  ^-irreducible  chain. 

From  this  simple  result  we  obtain  the  more  usable  condition: 

THEOREM  3.2  (Tweedie,  Ref.  [19],  Section  5):  If  <f>  has  an  atom  at  a then  a set  B con- 
taining a is  a test  set  if  for  some  integer  N and  some  6 > 0 

max  Pn(y,  (a})  ^8 

for  every  y 6 B. 

If  X - [0,  1,  2,  ...  ,)  and  {Jf„)  is  irreducible,  then  Theorem  3.1  implies  that  (0)  is  a test 
set;  Theorem  3.2  covers  the  result  that  (0,  1,  ...  A)  is  a test  set  for  any  finite  N.  See  Section 
10. 


4.  TEST  SETS  WHEN  THE  CHAIN  IS  WEAKLY  CONTINUOUS 

In  the  absence  of  the  discrete  type  of  behavior  of  Section  3 we  can  resort  to  a continuity 
condition  on  the  chain  to  identify  some  useful  test  sets. 

DEFINITION:  (iT„)  is  said  to  be  weakly  continuous  if  for  every  bounded  continuous  real 
function  / on  X 

P/(x)  - / P(x,  dy)  f(y) 

is  also  bounded  and  continuous.  Equivalently:  if  x„  -*  x then  P(x„,  ■)  — P(x,  •)  in  distribu- 
tion, i.e.,  Fx  (y)  — Fx(y ) at  every  continuity  point  of  the  latter  in  one  dimension. 

THEOREM  4.1  (Tweedie,  Ref.  [19],  Section  5):  If  {Y„}  is  a weakly  continuous  <t>- 
irreducible  chain  then  any  bounded  set  of  positive  (^-measure  is  a test  set  for  that  chain. 

Proving  weak  continuity  in  order  to  use  Theorem  4.1  may  often  be  difficult  if  the  chain 
has  a complicated  structure.  Since  we  feel  the  weak  continuity  condition  will  prove  the  most 
frequently  used  in  practice,  we  give  in  the  remainder  of  this  section  a routine  for  proving  weak 
continuity  which  reduces  the  evaluation  of  quite  complicated  models  to  a series  of  relatively 
simple  steps. 


t 
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In  many  models  the  chain  under  consideration  may  be  regarded  as  a secondary  process, 
derived  from  a primary  process  about  which  we  are  prepared  to  make  assumptions.  In  particu- 
lar, we  aim  to  carry  weak  continuity  of  the  primary  process  over  to  the  secondary  one.  That  is, 
suppose  the  transition  of  the  chain  {.V„)  (with  transition  probabilities  P(x,  •))  from  x to  x'can 
be  decomposed  into  a sequence  of  simpler  transitions:  from  x to  y|  according  to  a transition 
law  Q,(x,  •);  then  from  yx  to  y2  according  to  law  Q2Cv t-  and  finally  from  to  x 

according  to  the  law  Qk(yk-X,  ').  Then  weak  continuity  of  each  of  the  Q,(y,-X,  )’s  results  in 
the  same  property  for  the  P(x,  •).  See  Section  11  for  an  example  of  such  a decomposition.  To 
utilize  this  concept  we  give  the  following  simple  results. 

THEOREM  4.2:  If  the  transition  law  P(x.  •)  can  be  decomposed  as 

(4.1)  P(x.  A)  - J*...  J*  (?|(x,  dyx)  QjO't.  dy 2)  ...  Qk(yk- 1.  A) 
where  each  0,(y,_,,  •)  is  weakly  continuous,  then  P(x,  •)  is  also  weakly  continuous. 

PROOF.  If  / is  bounded  and  continuous,  and  Qk(yk-X.  •)  is  weakly  continuous,  then 
Q kf(y  k - 1)  is  continuous;  by  induction,  from  (4.1),  Pf(x)  = Qx  0i  • ■ ■ 0*/(x)  is  continu- 
ous, which  is  the  desired  result. 

When  applying  this  theorem  we  usually  make  the  transition  probabilities  Qj(y,-X.  ) as 
simple  as  possible  (see  Section  11).  Here  are  some  of  the  simple  forms  of  transition  probabili- 
ties for  which  weak  continuity  can  be  proved  fairly  easily; 

(i)  A transition  law  R (x.  •)  on  X will  be  called  degenerate  (at  h)  if  there  is  a function  h 
such  that  R(x.  •)  is  concentrated  at  h(x)  for  every  x 6 3C.  For  example,  in  Section  11  we  shall 

use  h(x,  y,  z)  - [(x  + y - z)+.  y.  z], 

(ii)  For  a vector  process,  the  transition  law  R (x,  •)  may  affect  only  a subset  of  the  coor- 
dinates of  x.  Formally,  suppose  3C  is  a product  3C)  x 3C2  of  two  spaces  of  lower  dimension,  and 
suppose  also  that  R(>  ■)  leaves  the  coordinate  unchanged  in  the  sense  that  (writing 
x = lx,,  x2l  in  the  obxious  manner)  there  is  a probability  measure  /?*(lx|.  x2).  •)  on  satis- 
fying 

(4.2)  / R([x„  x2J , d[yx,  y2])  f(yx,  y2)  = /?*Wx,,  x2],  dyx)  f(yx,  x2) 

for  every  bounded  measurable  / on  %.  In  this  case  we  say  that  R (x,  •)  acts  only  on  the  coor- 
dinate oft.  For  example,  the  transitions  in  Theorem  11.1  are  of  this  form. 


(iii)  When  % = X)  x 3C2,  one  of  the  coordinate  processes  may  form  a Markov  chain  in  its 
own  right.  That  is.  if  X„  - U,,,,  *„2],  then  {*,,,}  (say)  may  be  a Markov  chain.  In  this  case 
the  transition  law  of  the  X„x  chain  is  given  by 

(4  3)  /?i(x,.  A)  = R([x,.  x2],  A x 3(2) 

tor  any  » ( 1.  We  shall  identify  this  situation  by  saying  that  the  projection  on  the  3C,  coordi- 
•„>.  no><  is  Markovian.  Such  is  the  case  with  the  waiting  times  of  Section  1 1 . 

might  be  expected,  weak  continuity  for  transitions  of  any  of  these  forms  is  easier  to 


I 

I 


I 


I 


*i  »■  then  ti  i«  weakly  continuous  if  and  only  if  h is  continu- 
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(ii)  if  X = X,  x X2  and  R(x.  •)  acts  only  on  the  X,  coordinate  of  X,  then  it  is  weakly 
continuous  if  and  only  if 

K’&dx,.  x2])  = J*  R‘(U|.  x2],  dy ,) 
is  continuous  for  each  bounded  continuous  g on  X,; 

(iii)  if  X = X,  x X2  and  the  projection  onto  the  X,  coordinate  space  is  Markovian  (as  in 
(4.3)),  then  the  weak  continuity  of  R(lx,,  x2],  •)  implies  the  weak  continuity  of  R |(xt,  ■); 

(iv)  R (x,  ■)  is  weakly  continuous  if  and  only  if  Rf(x ) is  a continuous  function  of  x for 
every  bounded  uniformly  continuous  /on  X. 

PROOF:  (iv)  This  follows  directly  from  Theorem  2.1  (ii)  of  Billingsley  [2] 

(i)  Rf(x)  - f[h(x)]  which  is  continuous  in  x for  every  bounded  continuous  / if  and 
only  if  h is  continuous. 

(ii)  For  any  bounded  continuous  xon  Xi,  the  function  /(x,.  x ) - g(x,)  is  bounded  and 
continuous  on  X.  Since  Rf(x,.  x2)  - R'g(x,.  x2),  the  continuity  of  R'g  follows  from  that  of 

Rf 

Conversely,  suppose  R'g  is  continuous  for  every  bounded  continuous  g on  X,.  Let  / be 
bounded  and  uniformly  continuous  on  X.  If  x„  - |x„ ( x,2)  — x0  — (x0|.  x02l  as  n — 00  then 

| R/(x„)  - R/(xo)|  ^ J R*(x„.  dyt)\  /(>„  x,2)  -/(>,.  x02)| 

+ | f R'(xn,  dy |)  /O',.  x02)  - J R*(xo.  dy ,)  /O',,  x02)|. 

The  first  term  tends  to  zero  as  x„2  — x02  since  / is  uniformly  continuous;  the  second  term 
tends  to  zero  because,  for  fixed  X02.  i?0|)  “ /On  x0j)  is  continuous  and  R'g  is  then  continu- 
ous by  assumption. 

(iii)  If  g is  bounded  and  continuous  on  X,  then  /(x,.  x2)  - *(x,)  is  bounded  and  con- 
tinuous on  X — X,  * Xj-  Hence 

J fl,(x,.  </>,)  xOi)  ” J"  fl((x,.  x2).  dly |,  y2l)  /Oi-  ^2) 

is  a bounded  continuous  function  (not  depending  on  x2). 

5.  TEST  SETS  FOR  CHAINS  WITH  WEAKLY  CONTINUOUS  COMPONENTS 

Suppose  [P(x.  •))  is  a collection  of  substochastic  transition  treasures,  i.e.  for  each 
x,  P(x.  •)  is  a measure  on  5Fwith  P(x,  X)  ^ 1,  and  /*(-,  A)  is  a measurable  function  for  each 
A € J.  The  definition  of  the  iterates  P"(x.  A),  and  the  concepts  of  <&-irreducibility  and  weak 
continuity  of  the  (P(x,  )),-are  entirely  analogous  to  those  for  the  case  of  stochastic  transition 
laws.  In  the  same  way,  all  the  methods  for  proving  weak  continuity  given  in  Section  4 carry 
over  to  the  substochastic  case. 

Now  if  \P(x,  •))  is  the  family  of  transition  probabilities  for  a Markov  chain  {A'„},  then 
{ P(x,  ■))  is  said  to  be  a component  of  [P(x,  •))  if 

P (x.  A)  > P(x,  A)  for  every  x € X,  A € J. 

The  existence  of  suitably  well-behaved  components  can  be  used  to  identify  test  sets. 
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THEOREM  5.1  (Tweedie,  Ref.  [19],  Section  5):  If  the  chain  {*„)  has  a weakly  continu- 
ous, ^-irreducible  component,  then  any  bounded  set  of  positive  (^-measure  is  a test  set  for  the 
chain. 

The  use  of  components  is  illustrated  in  Section  12. 

Finally,  we  note  a simple  sufficient  condition  for  (A",)  to  have  a weakly  continuous  4>- 
irreducible  component.  Suppose 

P(x,  ■)  - a(x)  Px(x,  •)  + [1  - a(x)]  P2(x,  ■), 

where  a(x)  is  a continuous  function  with  0 < a(x)  < 1 for  all  x.  If  P |(x,  •)  is  ^-irreducible 
and  weakly  continuous,  then  P(x,  •)  — a(.x)  P,(x.  •)  is  a weakly  continuous  ^-irreducible 
component  of  {A’„),  no  matter  how  badly  behaved  P2(x.  ■)  is. 

6.  TEST  SETS  DEFINED  BY  SUBINVARIANT  MEASURES 

Our  final  method  for  finding  test  sets  is  the  hardest  to  use  in  practice,  but  it  will  prove 
necessary  in  Section  13  since  the  results  of  Sections  3,  4,  and  5 are  inadequate  for  handling  the 
inventory  model. 

If  (JO  is  a ^-irreducible  chain,  then  it  can  be  shown  (Tweedie,  Ref.  118])  that  there 
exists  at  least  one  nontrivial  tr -Unite  measure  m,  with  m » 6.  such  that 

(6.1)  ti(A)  > J n(dy)  P(y,A)  for  all  A € ?. 

Such  a n is  called  a subinvariant  measure. 

Unfortunately,  to  go  beyond  the  mere  existence  of  n is  often  tantamount  to  finding  an 
invariant  measure  (i.e.  one  for  which  the  inequality  in  (6.1)  can  be  replaced  by  an  equality), 
which  is  close  to  proving  ergodicity  directly.  Nevertheless,  the  following  is  useful  on  occasion. 

THEOREM  6.1  (Tweedie,  Ref.  [19],  Chapter  5):  Let  (O  be  a ^-irreducible  chain  with 
some  subinvariant  measure  fi.  Then  any  A € 5 with  0 < fi(A)  < «>  is  a test  set. 

7.  PROVING  <MRREDUCIBILITY 

Without  (6-irreducibility  the  chain  may  not  be  classifiable  at  all,  so  it  seems  worthwhile  to 
give  some  guidelines  for  establishing  irreducibility.  As  with  integer-valued  chains  though,  it 
may  often  be  necessary  merely  to  assume  (^-irreducibility,  or  to  give  conditions  which  are 
"grossly  sufficient"  for  ^-irreducibility  (see  Section  11).  There  are  two  cases  where  irreducibil- 
ity can  be  easily  checked: 

(i)  If  there  is  some  point  a 6 !C  which  can  be  reached  with  positive  probability  from  every 
point  in  the  space,  then  we  can  take  <b  to  consist  of  a single  atom  at  a.  See  Section  11  for 
example. 

(ii)  If  {JO  has  a ^-irreducible  component  (see  Section  5),  then  it  is  itself  also  <f>- 
irreducible.  Although  trivial,  this  observation  is  sometimes  quite  useful  since  it  may  be  easier 
to  work  with  the  iterates  of  some  component  of  the  chain,  rather  than  with  the  P"{x,  )’s  them- 
selves. 
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8.  RECURRENCE  FOR  CONTINUOUS-VALUED  CHAINS 

In  the  continuous  case  there  are  various  possible  definitions  of  recurrence:  see  Tweedie, 
Ref.  [19],  Section  3.  Here  we  shall  call  a (^-irreducible  chain  [Xn]  recurrent  if  there  is  a <£-null 
set  /V  such  that  for  all  x l N, 

(8.1)  Pr{TB  < °°|*0  = x)  = 1 

for  every  B € F with  <t>(B)  > 0.  The  null  set  A occurs  in  our  definition  because  one  can  prove 
that,  if  {X„)  is  not  recurrent  in  this  way,  then  it  is  transient  in  a natural  manner  (Tweedie,  Ref. 
[19],  Section  3);  such  a classification  fails  if  we  demand  that  (8.1)  hold  for  all  x and  all  flwith 
<t>(B)  > 0. 

Because  a dichotomy  between  transience  and  recurrence  exists,  we  can  again  hope  to  find 
individual  sets  that  will  help  classify  the  chain  itself.  If  the  validity  of  (8.1)  for  a particular  B 
implies  the  recurrence  of  the  chain  ( X„  ),  then  we  say  that  B is  a recurrence  test  set  for  that  chain. 
The  next  two  results  then  give,  respectively,  criteria  for  identifying  recurrence  test  sets,  and  a 
criterion  for  investigating  recurrence  using  these  sets. 

THEOREM  8.1  (Tweedie,  Ref.  [19],  Section  5):  Each  of  the  test  sets  (for  ergodicity) 
identified  in  Theorems  3.1,  3.2,  4.1,  5.1  and  6.1  is  also  a recurrence  test  set  for  the  same  class 
of  chains. 

THEOREM  8.2  (Tweedie,  Ref.  [19],  Section  10):  Suppose  [X„]  is  ^-irreducible.  Then  a 
sufficient  condition  for  recurrence  of  {X„}  is  the  existence  of  a non-negative  measurable  func- 
tion gand  an  increasing  sequence  [A„]  of  recurrence  test  sets  for  [Xn]  satisfying 

(8  2)  P(x.  dy)  £(>)  < g(x)  for  x € Ao 

(8  3)  ly  : *(y)  ^ n\  Q A„.  n - 1,  2,  ... 

Whilst  (8.2)  is  similar  to  the  condition  (2.6)  for  bounded  mean  return  times.  (8.3)  is  a restric- 
tion on  the  type  of  "test  function’  that  can  be  used  in  proving  recurrence.  In  general,  however, 

(8.3)  is  not  overrestrictive.  We  conclude  by  giving  the  most  common  form  of  use  of  Theorems 
8.1  and  8.2,  based  on  Theorem  5.1  and  the  test  function  g( x)  - x 

THEOREM  8.3  Suppose  X - [0,  <»)  and  [X„]  is  ^-irreducible  with  a weakly  continuous 
component.  Then  a sufficient  condition  for  [X„]  to  be  recurrent  is  the  existence  of  a constant  /3 
such  that 

(8.4)  J P(x,  dy)  y < x for  x ^ p. 

In  our  examples  we  concentrate  on  proving  ergodicity  rather  than  recurrence.  There  is  a 
pragmatic  reason  for  this:  the  ergodicity  criteria  of  Theorems  2.1  and  2.2  are  often,  rather 
surprisingly,  more  easily  verified  than  the  recurrence  criteria  of  Theorems  8.1  and  8.2.  For 
example,  we  often  impose  a condition  which  implies  that,  as  x — • oo, 

lim  sup  [ J,  P(x,  dy)  y - x]  < 0. 

which  immediately  gives  (2.9)  for  large  enough  x ; the  analogous  condition  to  give  recurrence 
would  seem  to  be 


lim  sup  | f P(x,  dy)  y - x < 0, 
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but  this  does  not  give  (8.4).  Hence  we  counsel  some  care  in  writing  down  the  "natural”  exten- 
sion of  ergodicity  conditions  as  recurrence  conditons.  However,  as  it  is  usually  quite  simple  to 
adapt  the  conditions  for  ergodicity  to  cover  the  recurrence  case,  we  shall  generally  leave  the 
formulation  of  these  extensions  to  the  reader. 

9.  INTRODUCTION  TO  THE  EXAMPLES 

The  three  steps  which  constitute  the  basis  of  our  technique  for  proving  ergodicity  were 
listed  near  the  end  of  Section  2.  We  now  illustrate  the  ways  in  which  this  technique  can  be 
applied,  by  proving  a number  of  ergodicity  results  for  a variety  of  Markov  chain  situations.  We 
do  not  pretend  that  these  results  are  in  any  sense  optimal;  indeed,  most  of  the  conclusions 
could  be  further  refined,  if  methods  suited  to  the  particular  example  were  used  (see  Kiefer  and 
Wolfowitz  (10],  or  Loynes  [11]).  Rather,  we  wish  to  demonstrate  the  essential  simplicity  of 
our  three-step  method.  In  each  case  the  proof  is  arranged  so  as  to  emphasise  this  point. 

10.  RANDOM  WALKS 

One  of  the  simplest  examples  of  a continuous-valued  Markov  chain  is  the  random  walk 
|T„]  on  (0,  «>)  defined  by 

(10.1)  - (•*„_,  + Y„)  * 

where  ( K „]  is  a sequence  of  independent,  identically  distributed  (-«>.  «)-valued  random 
variables.  The  next  result  is  extremely  well-known,  we  give  it  because  it  is  a straightforward 
example  of  the  techniques  espoused  above. 

THEOREM  10.1  If  El  I'd  < 0 then  |T„)  is  ergodic 

PROOF:  /.  <t>- irreduubihty:  We  show  that  *(K  the  point  mass  at  0.  is  a suitable  irreducibil- 
ity  measure.  Choose  a 6 > 0 such  that  Pr|  ^ -6)  “ y > 0.  Then  for  n ^ x8  ’, 

PrU, -OUo-  .v)  > y’>  0. 

II.  Test  sets:  Any  set  of  the  form  [0,  0|  is  a lest  set  This  can  be  proved  either  by  show- 
ing that  the  chain  is  weakly  continuous  or  by  applying  Theorem  3.2;  if  N > 06'  then,  as 
above. 

max  P"(v.  |0|)  > y v for  every  y € (0.  0]. 

»<  v 


III.  Boundedness  of  hitting  times:  Apply  Theorem  2.2. 

£[*,  - X0\X0  - x)  - £[max{  - *)] 

— E(K|)  as  x— by  dominated  convergence 

< 0. 

thus  EfAdlAfo  - x)  < x - « for  large  enough  x. 

11.  QUEUES 

Some  of  the  standard  queueing  results  follow  from  the  preceding  results  for  random 
walks.  In  this  section  we  consider  a slightly  more  complicated  situation:  a queue  with  waiting- 
time-dependent service  times.  Such  a queue  was  studied  by  Callahan  [3],  but  in  a rather 
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artificial  discrete  setting.  Tweedie  [20]  discussed  the  continuous  state  space  version  of  this 
model;  our  technique  for  proving  ergodicity  can  be  used  to  weaken  his  conditions.  Tweedie 
[21]  also  discusses  this  model  in  more  detail  using  a stronger  version  of  our  Theorem  2.1. 

Let  the  interarrival  times  {T,)  of  customers  in  a single-server  queue  be  independent  and 
identically  distributed  with  mean  n < °o.  Let  Wn  denote  the  waiting  time  of  the  n'h  customer, 
and  S„  the  service  time.  The  random  variables  ($„)  are  assumed  to  be  conditionally  indepen- 
dent of  each  other  and  of  the  T„' s,  given  the  relevant  waiting  times,  but  the  distribution  of  Sn 
does  depend  on  the  waiting  time  Wn: 

The  waiting  times  ( Wn ) are  defined  recursively  by 

(111)  + S._,-  r„)+; 

[ •*'„]  is  thus  a Markov  chain  with  state  space  [0,  °°).  Proving  the  <Hrreducibility  of  ( W„)  is 
difficult;  we  give  here  two  "grossly  sufficient"  sets  of  conditions  which  apply  to  two  different 
types  of  d>.  Both  of  these  conditions  are  weaker  than  those  proposed  by  Callahan  [3]  for 
discrete  state  space  chains,  or  Tweedie  [20]  for  the  continuous  state  space. 

(a)  For  all  w > 0,  and  some  8 > 0 and  y > 0,  — T.  < — 8|  W , - w]  > > 

This  makes  [Wn)  «„-irreducible.  ' *' 

(b)  There  exist  N > 0,  v > 0 such  that  FrlS,.,  - T„  > tj|  Wn_{  - «)  >0  for  w < N\ 
and  the  distribution  of  Sn-\  — T„  conditional  on  Wn  — <■>  has  a positive  density  on  (— »j,  0)  if 
•o  > N.  This  makes  ( ^-irreducible  with  as  Lebesgue  measure  on  (N  — rf,  N + tj). 

THEOREM  11.1:  Suppose  ( is  ^-irreducible  and  that  (i)  yj  ) is  weakly  continuous  in 


u; 

(11.2) 

(ii)  sup  £[S„|  W„  - <u]  < oo 

M 

and 

(11.3) 

lim  sup  £[S„|  W„  - <i>]  < fi. 

Then  ( W„)  is  ergodic. 

PROOF:  I.  <t>- irreducibility:  has  already  been  assumed. 

11.  Test  sets:  We  prove  that  ( Wn\  is  weakly  continuous,  so  that  [0,  0]  is  a test  set  for  all 
large  enough  values  of  0 (Theorem  4.1). 

Decompose  the  trivariate  chain  XR  - ( W„,  Tn)  as 
(11.4)  X„  — ( W„,  S,_„  Tn)  — (W„,  S„_y.  rB+I) 

01.5)  ~(W„,  S„,  rB+1) 

(116)  -[(^,  + S,-  Tn+l)\  S'.  r„+1] 

“ ^d+l' 

Then  (11.6)  is  degenerate  at  the  continuous  function  h(x,  y,  z)  - [(*  + y - z)+,  y,  z],  and 
so  is  weakly  continuous  by  Theorem  4.3  (i);  (11.4)  is  trivially  weakly  continuous  since  T„+ , is 
independent  of  T„  (and  X„)  — use  Theorem  4.3 (ii);  and  (11.5)  is  weakly  continuous  by 
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assumption  — use  hypothesis  (i)  and  Theorem  4.3(ii).  Thus  the  chain  is  weakly  continu- 
ous, by  Theorem  4.2,  and  hence  the  projection  ( W„\  is  also  weakly  continuous,  by  Theorem 
4.3(iii). 

III.  Boundedness  of  hitting  times:  Apply  Theorem  2.2  with  A =-  [0,  /3]  for  a large  enough 
value  of  j3. 

£[  Wx  - W0\  W0  - tul  - E[(w  + S0-  T,)+  - <o|  W0  « to] 

= £[max(S0  ~ T\,  — <o)|  fV0  “ »!• 

This  last  term  is  certainly  greater  than 

E[S0~  Tx iH'o-wl. 

but  exceeds  it  only  by  an  amount 

£[max{0,  f,  - S0  - <o}|  - a>)  ^ £[(r,-o>)+] 

— * 0 as 


Thus  it  suffices  to  show  that 

lim  sup  E[S0  - Til  M^o  “ <0- 

(Jj — »oo 

This  is  equivalent  to  (11.3). 

The  proof  of  ergodicity  for  the  usual  waiting  times  in  the  GI/G/1  queue  is  contained  in 
Theorem  11.1.  For  this  special  case  one  could  also  use  Theorem  3.2  for  establishing  that 
bounded  sets  are  test  sets  for  ergodicity. 


The  assumption  that  y„()  is  weakly  continuous  is  a natural  extension  of  the  case  where 
Sn  is  independent  of  however  it  does  not  cover  the  case  of  deterministic  service  times,  i.e., 
where  >„(•)  is  concentrated  at  the  point  y(co),  unless  y(u»)  is  a continuous  function.  We  can 
handle  some  discontinuous  y(w)'s  by  using  components. 


THEOREM  11.2:  Suppose  y „(•)  is  concentrated  at  y(«),  and  that  y(w),  although  not 
necessarily  continuous,  is  bounded  on  compact  sets.  Then  the  chain  ( Wr\  is  ergodic  under  the 
same  conditions  as  in  Theorem  11.1  except  that  (i)  should  be  replaced  by  (i)'-  T has  a density 
g(t)  which  is  not  concentrated  on  a bounded  set. 


PROOF:  We  find  a weakly  continuous  ^-irreducible  component  of  ( IF,).  Our  bounded- 
ness assumption  ensures  the  existence  of  a continuous  function  k(m)  with  *(«)  > y M for 
every  w.  Define  transition  measures  P(x.  •)  by  setting,  for  each  nonnegative  measurable  /, 

(11.7)  />  m/(0)  dl- 


so  that  in  particular,  for  any  set  A, 


(11.8) 


Since  git)  is  not  concentrated  on  any  bounded  set,  (11.8)  shows  that  P( to,  •)  is  one-step  «or 
irreducible.  Moreover,  for  any  /,  and  in  particular  for  bounded  continuous  /,  (11.7)  implies 
that  Plot,  f)  is  continuous  in  a>. 
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Since  P(u>,( 0,  °°))  - 0,  P(w,  A)  P(w,  A)  for  any  A C (0,  °°);  but  also 
P( co,  0)  = Pr(T  ^ a)  + y(w)  | W„  = *,) 


* dt 
= P(a>,  0). 

Thus  {P(a>,  •)}  is  a weakly  continuous  «o-irreducible  component  of  (/>(<u,  ■)).  The  rest  of  the 
proof  can  now  be  carried  out  as  in  Theorem  11.1,  but  this  time  [0,  /3]  is  a test  set  by  virtue  of 
Theorem  5.1  instead  of  Theorem  4.1. 

This  theorem  covers  the  type  of  service  times  considered  by  Sugawara  and  Takahashi 
117J;  conditions  (i)  and  (i) ' seem  to  cover  the  bulk  of  the  likely  models  for  waiting  times. 

Since,  in  these  two  nonoverlapping  cases,  assumption  (ii)  gives  ergodicity,  it  may  seem 
that  this  assumption  alone  is  sufficient  for  ergodicity,  and  that  steps  I and  II  are  unnecessary. 
The  following  somewhat  artificial  example  shows  that  this  is  not  the  case;  boundedness  of  the 
hitting  times  on  compact  sets  does  not  give  ergodicity  if  the  chain  is  sufficiently  incompatible 
with  the  topology  of  the  space. 

Example.  Suppose  the  arrivals  in  the  waiting-time-dependent  service-time  model  above 
are  deterministic,  with  interarrival  times  of  length  1.  The  waiting  times  will  take  on  values  in 
0,  1],  and  the  conditional  distribution  of  the  service  time  S are,  for  W € H -> 

'M  4 

/Vis  - 1 + (n  + l)-‘  - n-‘|  W-  n 'i  - a,>  0. 

Pr[S  - 2 - /»-'|  W - /!-')  - 1 - a„  > 0. 

and  for  u € [0.  1J\//, 

/V{S-1  - 1, 

Pr[S  — 2|  — 0}  — 1. 

Hence  { W„)  is  ^-irreducible  with  <f>  as  counting  measure  on  the  set  H,  and  in  at  most  two  steps 
from  any  &»,  ( tVn]  takes  values  in  this  set  ("two-step  irreducibility"). 

Now  for  all  a ■ > *, 

E[  Wx  - W0\  W0  - oj]  < -min{«,  [a„(»  + l)-1  + 1 - a„  - «"'l:  n < *-1}. 

Hence  if  — • 1 sufficiently  fast,  { W„)  will  have  mean  drift  towards  each  set  of  the  form  [0,  «]. 
But  if  11  a,  >0,  then  the  chain  has  no  stationary  distribution  (intuitively  such  a distribution 
would  be  concentrated  at  zero,  and  this  cannot  happen). 

12.  DAMS 

We  consider  two  generalizations  of  the  Moran  model  of  a dam  with  infinite  capacity  (see 
Prabhu,  Ref.  (15],  Chapter  6).  The  content  Z„  of  the  dam  at  time  n is  determined  by  a 
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sequence  (A",,}  of  independent  identically  distributed  inputs  and  the  fixed  quantity  m (=  amount 
released  per  unit  time): 

(12.1)  Z«+i  = Z„  + X„-  min(«,  Z„  + X„). 

We  retain  the  same  input  scheme  but  replace  the  deterministic  release  rule  by  a random  release 
rule. 

The  first  model  incorporates  a "release  function"  R (•);  replace  m by  R (Z„)  to  define 
Z„+1  = Zn  + X„-  min {R  (Z„),  Z„  + X„) 

(12.2)  - IZ„  + X„-  /?(Z„)]+; 

cf:  the  general  release  rule  for  the  continuous-time  model  of  Moran  [13]. 

THEOREM  12.1:  Suppose  the  random  variables  Xn  have  finite  mean  p > 0 and  a density 
function  g which  is  positive  at  each  point  of  (0,  °°).  Then  the  Markov  chain  \Z„]  is  ergodic  if 

(i)  R (•)  is  continuous, 

(12.3)  (ii)  lim  inf  /?(u)  > p. 


PROOF:  We  shall  produce  a ^-irreducible  weakly  continuous  component  of  the  chain 
{Zj.  This  will  not  only  establish  the  0-irreducibility  of  the  chain  (Section  7)  but  will  also 
prove  that  bounded  sets  of  positive  ^-measure  are  test  sets  (Theorem  5.1);  this  takes  care  of 
steps  I and  11  of  the  method  for  proving  ergodicity. 

For  any  nonnegative  function  /, 

£[/(Z,)|Z0-z]  - f~flz  +*  - R(z)l  *(*)  dx 

3*  J*  /[z  + x - R(z)l  g(x)  dx 

- P(z.  /),  say. 

By  construction,  the  substochastic  transition  laws  P(z,  •)  defined  by  the  above  expression  con- 
stitute a component  of  (the  transition  laws  of)  the  chain  [Z„). 

I.  <t>- irreductbiUty  of  the  component:  Because  of  (ii)  there  exists  a constant  u0  > p such 
that  R (u)  > p + c whenever  «>«,>,  for  some  positive  «.  Let  <t>  be  Lebesgue  measure  on 
[u0,  oo].  Since,  by  a change  of  variable, 

P(z,  /)  ” f ./(<*>)  sfw  + R(z)  — z]  dot, 

•'h-R(r)l  + 

and  g is  everywhere  positive  on  (0,  °°),  it  follows  that  P(z,  A)  > 0 if  z ^ u0  and  </>(A)  > 0. 
Moreover,  since 


P"+'(z.  A)  > f“°  P"(z.  du)  P(u,  A), 

•'O 

0-irreducibility  will  follow  if  P"(z,  [0,  uj)  > 0 for  every  z > u„,  where  n depends  upon  z. 
But  if  z > u0  then  R (z)  ^ p + «,  and  so 

P[z,  (z  — p,  z — p/2)]  > f /(w)  g[o>  + R (z)  — z]  do> 

'*2-11 
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II.  Test  sets:  We  prove  that  P(z,  •)  is  weakly  continuous  so  that,  by  Theorem  5.1,  any 
bounded  set  with  positive  ^-measure  will  be  a test  set. 

Now  if  / is  bounded  and  continuous,  and  if  z„  — * z,  then 

P(z„.  /)  - /(R(  )__  ]+f\zn  + x - R (z„) ] g(x)  dx 

il*u,-2|+/[z  + * “ *(z)1  «(x)  dx 

by  dominated  convergence  since  both  (R(z)  - z)+  and  f(z+x-  R(z ))  are  continuous  func- 
tions of  z. 

HI.  Boundedness  of  hitting  times:  Use  Theorem  2.2  with  A = [0,  /3]  for  /3  large  enough. 
E[Z]  - Z0|Z0=  z]  - £[max{A'1  - R(z),  z]  1 

^ £[max{JTi  — fi  — e,  — z}]  if  z ^ u0 
— £[-Ti  — m — «]  as  z — oo,  by  dominated  convergence 


We  remark  that  essentially  this  general  release  model  has  been  considered  in  the  much  more 
difficult  continuous-time  case  by  Cinlar  and  Pinsky  [5]  and  Harrison  and  Resnick  [8].  Their 
methods  are  by  necessity  much  more  sophisticated  than  the  straightforward  techniques  that  we 
can  give  in  the  discrete-time  case. 

The  conditions  of  Theorem  12.1  can  be  modified  slightly  to  give  other  conditions  for 
ergodicity: 

(a)  It  suffices  for  /?(w)  to  be  greater  than  n + t for  every  w ^ 0;  no  conditions  are 
needed  for  the  chain  [X„).  In  this  case  [Z„\  is  «o-irreducible.  The  standard  fixed  release  rule  of 
(12.1)  is  covered  by  such  an  assumption. 

(b)  Suppose  the  common  density  function  g of  the  2f„’s  is  bounded  below  by  a strictly 
positive,  decreasing,  continuous  function  g\  and  that  R(  ) is  bounded  above  by  a continuous 
function  /?'(■)  and  is  bounded  below  on  [w0,  »)  by  a positive  continuous  function  /?"(•). 
Then  the  method  of  proof  of  Theorem  12.1  goes  through  using  the  component 

P*(z,  /)  - f s'lw  + R'(.z)  - z]d( o; 

notice  that  P*(z,  ■)  < P(z,  •). 

Further  generalizations  of  the  basic  dam  model  are  possible.  We  leave  to  the  reader  a 
proof  of  the  following  random-release  result,  noting  only  that  the  assumptions  ensure  «o* 
irreducibility. 

THEOREM  12.2:  If  {K„|  is  a sequence  of  independent,  identically  distributed,  non- 
negative random  variables  which  are  independent  of  U„),  we  can  define  the  random-release 
dam  model  {Z„}  by 

Z„+,  - (Z„  + Xn  - Yn)+. 

If  E(X„)  < E(Yn)  then  { Z„\  is  ergodic. 
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13.  INVENTORIES 

The  standard  inventory  model  poses  a more  difficult  problem  since  the  transition  laws 
exhibit  distinctly  noncontinuous  behaviour  — even  the  existence  of  a weakly  continuous  com- 
ponent would  require  intuitively  unnatural  assumptions.  However,  the  chain  can  be  analyzed 
by  means  of  the  result  in  Section  6. 

Let  [X„]  be  the  s-S  inventory  model  (Prabhu,  Ref.  [15],  Chapter  5): 

- f „ if  s < A„_,  < S, 

Xn  = S-t„  if  A„_,  s, 

where  the  £„  are  independent,  identically  distributed  non-negative  random  variables  with  com- 
mon distribution  law  F(  ) and  0 < s < S.  The  chain  (A,)  has  state  space  (-<*>,  S]. 

THEOREM  13.1:  Suppose  F{0)  < 1.  Then  {X„)  is  ergodic. 

PROOF:  I.  <t>-irreducibility:  Clearly  </>(•)  = P(S,  •)  is  a suitable  irreducibility  measure. 

II.  Test  sets:  Let  n be  a subinvariant  measure  for  the  chain.  The  subinvariant  equation 

(6.1)  can  be  written  as 

(13.1)  is(dy)  P(y,  A)  + /n(-°°.  si  F(S,  A), 

so  if  A is  any  set  with  P(S,  A)  > 0 and  n(A)  < °°  then  /*(-«>,  s]  < <».  By  assumption, 

Pr[t„  > 0)  > 0 hence  Pr[£ „ ^ «)  > 0 for  some  « > 0.  ■ Thus,  if  m(_°°.  s]  “ 0 then  by  tak- 

ing A - (—oo,  x]  in  (13.1)  we  would  obtain  m(s.  s + e]  — 0.  Repeated  application  of  this 
argument  would  lead  to  the  conclusion  that  S]  — 0,  which  contradicts  the  assumed 

nontriviality  of  fi.  Thus  0 < /&(-oo,  $]  < oo,  and  so  by  Theorem  6.1,  (— oo,  s]  is  a test  set. 

III.  Boundedness  of  hitting  times:  Use  the  test  function  g(x)  - (x  + D)+  where  D is  any 
fixed  positive  quantity.  Take  A - (-oo,  s]  as  the  test  set.  Then  for  x € Ac  — (s.  S], 

E[g(Xx)\X0~  x]  « J(Q  oo)  (x  - y + D)+  F(dv) 

-■x  + D-  f x + D - (x  - y + D)+  F(dy) 

- x + D - f min|x  + D,  >}  F(dy) 

•riO.  «•) 

< X + D - J*  minis  + D,  y)  F(dy). 

This  last  integral  is  strictly  positive  and  independent  of  the  value  of  x.  Notice  also  that 
^[^(A^lAo  - x]  < S + D for  every  x.  Thus  Theorem  2.1  can  be  used  to  prove  that  |A„)  is 
ergodic. 

By  a repetition  of  the  argument  for  step  II  it  could  be  shown  that  /s(- oo,  S]  < oo.  Thus 
the  whole  state  space  itself  is  a test  set.  For  this  test  set  the  mean  drift  condition  of  Theorem 
2.2  is  trivially  satisfied  so  ergodicity  follows  as  before.  We  should  remark  that  finiteness  of  a 
subinvariant  measure  can  also  be  shown  to  be  equivalent  to  ergodicity,  by  more  direct  methods. 

14.  FEEDBACK  MODELS 

Let  ( Y„)  and  [Z„]  be  mutually  independent  sets  of  independent  identically  distributed  ran- 
dom variables,  and  let  b be  a positive  constant.  A general  form  of  the  feedback  model  |An)  is 
specified  by 
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(14.1) 


l»+i 


X„  + Y„  if  |jf,|  ^ b, 

X„  + Y„  - Z„  if  Xn  > b,  and 
Xn+Y„+  Z„  if  Xn  < -b. 


This  model  was  introduced  by  Bellman  [1],  with  the  Z„'s  constant.  Calton  and  Rogers  [4]  also 
analyzed  a deterministic  form  of  the  model,  using  results  similar  to  our  Theorems  2.1  and  8.2. 


The  model  described  by  (14.1)  is  not  weakly  continuous,  and  so  we  look  for  a weakly 
continuous  component.  Let  Q{(A)  = Pr\Y„  6 A],  Q2(A)  — Pr[Y„  — Z„  € A)  and 
Q3(A)  = Pr[  Y„  + Z„  € A ),  and  write  Q for  the  largest  measure  such  that  0(A)  ^ 
min[£)|(y4),  Q2(A),  Q3(A)]  for  every  Borel  set  A.  A sufficient  condition  for  Qto  be  nonzero 
is  that  Y„  admits  a density  g(y)  satisfying,  on  some  set  of  positive  Lebesgue  measure, 

oo 

min[*0>),  f_"  g(y  + «)  dPr[Zn  ^ a),  g(y  - a)  dPr{Z„  < a)]  > 0. 

In  the  deterministic  model,  for  example,  this  will  occur  if 

minis (>0,  g(y  - k),  g(y  + A)]  > 0 

where  the  Z„'s  take  the  constant  value  k ; for  this  it  would  suffice  to  have  g(y)  > 0 on  some 
set  of  the  form  (—k  - 8,  k + 8). 


THEOREM  14.1:  Suppose  Q dominates  Lebesgue  measure  on  some  interval  (— rj,  -r)).  If 
|£(K„)|  < E(Z„)  then  {*„}  is  ergodic,  whilst  if  |£0'„)|  < E(Z„)  then  {A'J  is  recurrent. 

PROOF:  Define  P(x,  A)  •»  Q(A  — x).  Then  by  definition  (P(x,  •)}  is  a component  of 
{ P(x , •)),  and  it  is  ^-irreducible  with  <f>  as  Lebesgue  measure,  from  our  first  assumption  (which 
is  again  "grossly  sufficient";  see  Revuz,  Ref.  [16],  Section  3.4,  for  less  practical  but  finer  condi- 
tions). Moreover,  P(x,  ■)  is  weakly  continuous:  if /is  bounded  and  continuous  and  x„  — x, 

P(x„,  /)  - f/(x„  + ^)  Q(dy) 

-//<-  + y)  Q(dy)  by  dominated  convergence 

- P(x,  /). 

It  is  simple  to  show  that  the  second  assumption  implies  that  £[|A',||A'o  - x]  < x - « for  all  x 
outside  some  bounded  region  ( -M , A/),  whilst  £[|Af,||Af0  - x]  < M + £|  L0|  + £|Z0|  < ~ if 
x €(— M,  M).  Ergodicity  follows.  Similarly  the  third  assumption  enables  us  to  use  Theorem 
8.2,  whence  the  recurrence  result. 


Using  techniques  similar  to  those  of  Section  12,  we  could  handle  feedback  models  with 
more  general  feedback,  e.g.,  we  could  replace  (Z„)  in  (14.1)  by  R(X„).  A typical  set  of  condi- 
tions for  ergodicity  would  then  be  that  R(  ) is  continuous  (to  establish  weak  continuity),  that 
Y „ has  positive  density  everywhere  (to  give  <^-irreducibility  with  <f>  as  Lebesgue  measure)  and 
that  lim  inf  R(x)  > E(Y„)  and  lim  sup  R(x)  < E(Y„)  (to  establish  mean  drift  towards 

oo  x 

some  bounded  interval). 
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ABSTRACT 

In  this  paper  we  consider  the  problem  of  maximizing  the  sum  of  certain 
quasi-concave  functions  over  a convex  set.  The  functions  considered  belong  to 
the  classes  of  functions  which  are  known  as  nonlinear  fractional  and  bi- 
nonlinear  functions.  Each  individual  function  is  quasi-concave  but  the  sum  is 
not.  We  show  that  this  nonconvex  programming  problem  can  be  solved  using 
Generalized  Benders  Decomposition  as  developed  by  Geoffrion. 


INTRODUCTION 

In  this  paper  we  consider  the  application  of  the  Generalized  Benders  Decomposition 
(GBD)  as  developed  by  Geoffrion  [4]  to  certain  nonconvex  programming  problems.  Before 
showing  how  this  is  done,  however,  we  briefly  summarize  the  application  of  GBD  to  a problem 
whose  form  is  similar  to  the  one  of  interest. 

The  variables  of  our  problem  are  of  two  forms,  y€£"and  x€£m.  The  problem  we  wish 
to  solve  is  given  by  problem  P : 


P:  maximize  z = fix) 


subject  to  G iy,x)^0, 

x € X.yt  Y. 

Of  interest  here  are  problems  where  the  functions  Giy.x)  are  not  concave  in  x and  y jointly, 
but  fixing  x renders  them  so  in  y and  fixing  x yields  the  same  for  y.  The  sets  X and  Y are  taken 
to  be  convex  sets.  The  function  fix)  is  assumed  concave. 

Goeffrion  showed  that  undei  certain  conditions  problem  P could  be  solved  for  a global 
maximum,  even  though  it  appears  at  first  glance  to  be  a nonconvex  programming  problem.  To 
achieve  this  result,  Goeffrion  extended  the  work  of  Benders  [2]  to  include  nonlinear  program- 
ming problems.  We  now  describe  the  (GBD)  method  for  solving  problem  P.  A knowledge  of 
the  method  is  not  necessary  but  may  be  helpful.  The  procedure  is  represented  by  the  following 
steps: 

STEP  1:  Let  a point  y€  Tbe  known.  Solve  the  following  optimizatioi.  problem: 

! 

PI:  maximize  z - fix) 
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subject  to  G ( y,x)  ^0, 

x 6 X. 

The  value  of  z in  the  solution  to  PI  is  a lower  bound  on  the  optimal  solution  to  problem  P ; 
call  it  LB.  In  subsequent  steps  it  may  be  necessary  to  solve  PI  several  times.  The  value  of  LB 
is  always  given  as  the  greatest  of  the  values  of  z in  solutions  to  problem  PI.  Denote  by  u >0 
Kuhn-Tucker  multipliers  associated  with  the  constraints  G(y,x)> 0 in  the  solution  to  problem 
PI.  We  assume  that  such  multipliers  are  given  by  the  method  used  for  solving  problem  PI. 
We  also  assume  that  for  any  y € Y there  exists  an  x € X such  that  G(y,x  ) ^0.  Thus,  problem 
PI  will  always  have  a feasible  solution.  Now  go  to  Step  2 with  p—1  and  . 

STEP  2:  Solve  the  optimization  problem 

P1A:  maximize  y„ 

subject  to  y0  < maximum  {/(x)  + uJg(y,x)),  J - 1 p , 

jc€  X 

y € Y. 

Let  (y0,y)  solve  problem  P1A.  Let  UB  - y0,  where  UB  is  now  a known  upper  bound  on  the 
value  of  the  optimal  solution  to  problem  P. 

STEP  3:  Return  to  problem  PI  with  the  solution  yand  solve  it.  If  LB'zUB-t  (where  « 
is  a prechosen  number),  terminate  with  yand  the  corresponding  solution  to  problem  PI  as  the 
solution  to  problem  P.  Otherwise,  increase  p by  one  and  put  m'’-  u and  return  to  Step  2.  We 
now  point  out  various  aspects  of  the  algorithm. 

Note  that  in  Step  1 the  optimization  problem  is  essentially  problem  P with  the  vector  y 
fixed.  The  lower  bound  created  at  this  stage  may  not  increase  at  each  iteration  of  the  algorithm^ 
We  assume  that  problem  PI  satisfies  some  sort  of  regularity  assumption  so  that  the  vector  u 
exists. 

Step  2 of  the  method  is  the  most  difficult  to  perform.  Problem  PI  A is  essentially  the  dual 
of  problem  P.  The  value  of  y0  in  the  solution  to  problem  PI  A will  be  strictly  decreasing,  since 
each  problem  is  more  constrained  than  its  predecessor.  We  thus  see  that  the  value  of  UB  will 
decrease  at  each  iteration  until  the  method  terminates.  The  solution  of  problem  PI  A will  gen- 
erally be  in  two  steps.  First,  compute  the  maximum  on  the  right-hand  side  of  the  constraints 
corresponding  to  u".  Second,  solve  problem  PI  A for  the  values  of  (y0,y).  The  first  of  these 
steps  is  the  most  difficult,  unless  the  maximum  can  be  carried  out  independently  of  the  vector 
y.  Note  that  once  this  maximization  is  carried  out,  the  second  step  is  straightforward  if  the 
function  G(y,x)  is  assumed  to  be  concave  in  yfor  fixed  x. 

Termination  of  the  method  in  a finite  number  of  steps  (for  « > 0)  is  provided  by 
Geoffrion  in  the  following  theorm,  modified  slightly  to  correspond  to  problem  P. 

THEOREM  (Geoffrion,  Ref.  (41):  Assume  that  X and  Y are  compact  convex  sets  and 
that  for  every  y € Y there  is  an  x * € X such  that  G(y,x*)  > 0.  Also,  assume  that  the  func- 
tions /and  G are  concave  on  X for  every  fixed  y € Y,  and  that  problem  PI  satisfies  a regularity 
assumption  guaranteeing  multipliers  u for  every  y € Y.  Then,  for  any  * > 0,  the  GBD  pro- 
cedure terminates  in  a finite  number  of  steps. 

In  the  following  sections,  we  shall  show  that  the  assumptions  of  the  above  theorem  are 
satisfied  by  two  problems  which  are  equivalent  to  nonconvex  programming  problems. 
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NONLINEAR  FRACTIONAL  PROGRAMMING 


In  this  section  we  consider  a special  form  of  problem  P.  While  the  constraint  set  is  a con- 
vex set,  the  objective  function  is  not  concave,  and  thus  maximization  may  lead  to  locally 
optimal  but  globally  suboptimal  solutions.  The  problem  takes  the  form: 

m 

P:  maximize  z - '£f,(y)/cl(y) 
i-i 


subject  to  y € Y 

Once  again  we  assume  that  y € E"  and  that  T is  a convex  set.  Each  term  of  the  objective  func- 
tion is  the  ratio  of  /,  O'),  a concave  function,  to  c,(y),  a linear  function  in  y.  We  assume  that 
cA.y)  > 0 snd  f,(y)  ^ 0 for  y€  Y,  /—I m.  Such  functions  belong  to  the  class  of  func- 

tions which  Mangasarian  [5]  terms  nonlinear  fractional  functions. 

It  is  well  known  that  each  function  /,(y)/c,(y)  is  a quasi-concave  function,  in  the  sense 
that  for  constant  x,  > 0,  the  set  fAy)/c,(y ) > x,  is  a convex  set.  We  shall  use  this  result  to 
formulate  another  optimization  problem  which  is  equivalent  to  problem  P,  in  the  sense  that 
both  problems  have  the  same  optimal  solution.  We  will  apply  the  GBD  procedure  to  the 
equivalent  problem. 

To  formulate  the  equivalent  problem,  we  make  note  of  the  fact  that  it  is  always  possible 
to  determine  a value  of  x,"  so  large  that  the  inequality  has  no  solution  for  y € Y.  If  this  is  not 
true,  then  the  problem  would  have  an  infinite  solution.  In  practice  one  could  determine  x,"  by 
maximizing  each  function  f\y)/c,(y)  over  Y.  (Theoretically,  one  must  assume  fl(y)/ci(.y)  is 
pseudoconcave  to  perform  this  maximization  [1,5],  but  in  actual  practice  this  rarely  causes 
difficulties.)  We  thus  assume  that  one  can  obtain  the  numbers  x",  /—l,  ...  ,m. 

Given  the  above  we  formulate  problem  P'  : 

m 

P':  maximize  z - £ x, 

/-I 


subject  to  fi(y)-xi  c,  (>-)  > 0,  /-I  ...  ,m, 

y€  y(0<x,<x",/-l, 

We  see  that  the  optimal  solutions  to  problem  P and  problem  P'  are  the  same.  For  notational 
purposes  we  denote  the  set  X “ {x,|0  ^ x,  ^ x“  /“l,  ...  ,m ).  Comparing  the  components  of 
problem  P'  to  the  convergence  theorem  for  the  GBD  procedure  we  find  that  the  sets  Y and  X 
are  compact,  and  that  for  any  >€  Y there  exists  x‘  € X such  that  G{y,x*)  - f,(y)~ 

x,  *c,(y)  >0  , /— 1 m.  We  note  also  that  /,  (y)-x,  c,(y)  is  concave  in  x,  for  fixed  y.  Thus, 

if  we  can  perform  the  steps  of  the  method  we  can  guarantee  that,  for  any  c > 0,  the  GBD  pro- 
cedure will  yield  the  optimal  solution  to  problem  P'  in  a finite  number  of  steps. 

Applying  the  GBD  procedure  to  problem  P'  we  must  first  solve  at  each  stage  the  problem 
P'  1 with  y fixed  at  y € K, 

m 

P'l  maximize  z - £x, 

• <-i 
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subject  to  /,(j')-x,c/(>')>  0 , / — 1 m, 

x€  A'. 

It  is  easily  seen  that  problem  P'  1 is  in  fact  equivalent  to  m linear  programming  problems  in  one 

variable,  and  thus  has  the  closed  form  solution  x,  — f\y)/c,(y),i-\ m.  Since  P'  1 is  a 

linear-programming  problem,  the  Kuhn-Tucker  multipliers  always  exist  and  in  fact  have  closed 
form  solution  u,  -1/c,  (y),  /— 1 . ...  ,m. 

The  first  time  problem  P'lA  is  solved  it  will  have  the  form 

P'lA:  maximize  ya 


m 

subject  to  maximum  £x,  + £w,  [fXy^-XiC^y)), 


xi  x 


i-i 


y € Y. 


Because  of  the  form  of  X this  yields  the  problem 
P'lA:  maximizes 


subject  to  y0<  ip/f/(y)+  £ maximum  {x,[l-M,c,(.y)])> 

i- 1 /-I 

>-€  Y. 

For  a typical  term  on  the  right-hand  side  of  the  constraint,  the  maximization  will  be  given  by  x, 
- 0,  if  1 — u,  c,(y ) < 0,  and  x,  - x,"  if  1 — m,  c,(>>)  > 0.  Of  course,  when  1 - m,c,(>>) 
-0,  the  value  of  x,  is  of  no  consequence.  Thus,  in  order  to  solve  problem  P'lA,  one  must 
solve  2m  problems  considering  all  possible  si-jns  of  the  terms  1 - i7,  c,(y).  For  example,  when 
the  enumeration  is  performed  one  might  start  with  1 - u,  c,(y)  < 0 for  all  / and  solve  the 
problem 

P’  1A  : maximize  y„ 

m 

subject  to  y0  < 2,u,/,(y). 

i-i 

l-ulc,(y)£0,i“l,  ...  ,m, 
yt  Y. 

This  problem  is  a concave  programming  problem,  since  u,  > 0,  c,{y)  is  linear,  and  f,(y)  is 
concave  for  /-l,  , m.  We  might  next  consider  the  problem  with  1 - m,  C|(.y)  > 0 and  all 

other  conditions  unchanged. 

This  gives  the  problem 

P'  1A  : maximize  y0 


GENERALIZED  BENDERS  DECOMPOSITION 


477 


t. 

subject  to  y0<  £ uJlym-U'C^y)], 

/-I 

t 

This  is  also  a concave  program  due  to  the  linearity  of  c^y). 

In  order  to  solve  problem  P'lA  one  must  first  solve  all  2m  cases  and  take  as  (y0y)  the 
solution  with  the  largest  value  of  y0.  This  becomes  an  upper  bound  on  the  optimal  solution  to 
problem  P\  Of  course,  this  will  be  quite  cumbersome  if  m is  large,  but  many  practical  prob- 
lems are  still  within  the  range  of  solvability  since  it  is  the  number  of  quasi-concave  functions  in 
the  sum,  rather  than  the  number  of  variables  in  the  problem,  that  determines  the  degree  of 
difficulty  of  the  problem. 

There  are  two  additional  points  of  difficulty.  Once  problem  P'lA  has  been  solved,  it  is 
necessary  to  carry  the  contraints  on  the  quantities  1 — u,c,(y)  forward  to  the  next  formulation 
of  P 1A.  Thus,  at  each  step  of  the  method  the  constraints  on  P'lA  grow  larger.  Since  they  are 
linear,  this  may  not  increase  the  difficulty  of  P'lA  too  much.  A more  serious  problem  occurs 
when  two  or  more  different  solutions  at  any  one  stage  tie  for  having  the  maximum  value  of  y0. 
In  this  case,  it  is  necessary  to  carry  all  these  solutions  to  the  next  stage,  solve  problem  P'  1 for 
each  y (unless  they  are  the  same)  and  then  solve  P'  1A  for  each  combination.  Thus,  if  k solu- 
tions tie  in  the  solution  to  problem  P'lA  at  any  stage,  it  might  be  necessary  to  solve,  at  a 
minimum,  k2n  programming  problems  the  next  time  problem  P'lA  occurred. 

The  combinatorial  nature  of  the  solution  procedure  may  at  first  seem  depressing.  It 
should  be  pointed  out,  however,  that  it  has  been  shown  (31  that  zero-one  integer  programming 
problems  can  be  formulated  as  a special  case  of  this  problem.  Viewed  from  this  perspective,  it 
is  not  surprising  that  the  effectiveness  of  the  procedure  deteriorates  exponentially  in  the 
number  of  functions  in  the  sum.  Keeping  this  in  mind,  we  present  an  example  of  the  method 
in  the  next  section. 

AN  EXAMPLE  PROBLEM 

Consider  the  following  problem: 

P:  maximize  z-y,+l/y,+y2  + y2/l-y2 

subject  to  y€  Y - {yi,y2|y,+y2>l, 

>’i+>’2<2,y1>0,y2>0}. 

A geometric  view  of  the  problem  with  the  value  of  z given  at  each  extreme  point  appears  in 
Figure  1.  It  appears,  from  the  extreme  points,  that  there  are  two  local  maxima.  It  should  be 
pointed  out,  however,  that  there  is  no  guarantee  that  the  global  maximum  occurs  at  an  extreme 
point.  Taking  x"  — xj  -»  2,  we  formulate  problem  P'  as 

P':  maximize  z - X!+x2 


J 


subject  to  yi+l-x,(y,+y2)  >0, 

yi-x2O-y2)>0. 

yZY  - {y,,y2|y,+y2>l.y,+y2<2,y,>0,y2>0). 
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VI 


and 

x€  X “ ( xj,X2|O^Xi^2, 0^x2^ 2 }. 

To  begin,  we  pick  the  local  optimum,  yx  - 1,  y2  “ 0,  and  get  problem  P'  1: 

P' 1:  maximize  z-X!+x2 

subject  to  X|<2,3x2<0,x€  X,  which  yields  x,  - 2,  x2  - 0,  z - 2,  so  that  LB  - 2.  We 
get  multipliers  u | *■  1,  u2  *■  1/3.  Problem  P'lA  has  the  form: 

P'lA:  maximize  y0 

subject  to  v0<maximum  (x1+x2+[>>1+l-Xi(>'i+>'2)l+l/3[>'2-x2(3-y2)]}, 

0<*1<2 

0<*j<2 

>€  Y. 

Simplifying,  we  get  the  problem 
P'lA:  maximize  y0 
subject  to  1 

+(l/3)y2+l+  maximum  lxill-(>'j+>'2)]+x2ll-(l/3)(3->2)l), 

0<*,<2 

0«Uj<2 

>€  Y. 

We  must  solve  four  problems: 

1:  maximize  y0 
subject  to  y„  < -y  |->  2+3 , 

y,+y2<l.(l3'2^0.3'€  Y, 
with  optimal  solution  y\  - l.j'j  “ 0,y„  - 2. 
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2:  maximize  y0 

subject  to  >’„<->'|-(5/3)>’2+3, 

1.0/3) >3^0, >€  Y, 


with  optimal  soluton  yx  - 1.  y2  - 0,  y„  — 2. 

3:  maximize  y„ 

subject  to  y„^  yt  + y2  + 1, 

Mi  + +2  ^l.(l/3)>j  20.y€  Y. 


with  optimal  solution  y , - 2.>'2  - 0,y„  - 3. 

4:  maximize  y„ 

subject  to  >„<>i+(l/3)jr2+l. 

J'i+3'2^1.O/3)^2<0,>€K( 

with  optimal  solution  yi  - 2. >2  - O,^  - 3. 


Problems  3 and  4 have  the  same  value  of  y„  (we  take  this  solution  to  problem  3 to  aid  the 
explanation,  although  the  solution  yx  - 0.  - 2^0-3  has  the  same  value),  and  thus  ya~ 3 
serves  as  an  upper  bound  on  the  value  of  z in  problem  P'.  We  thus  proceed  to  problem  P\ 
with  LB  — 2,  UB  — 3,  and  the  solution  — 2.>>2  “ 0. 

P'l:  maximize  z - x(+x2 

subject  to  2x,  < 3,  3x2  <0,  x£X.  The  solution  gives  x,  - 3/2,  x2  - 0.  and  z - 3/2  so 
that  LB  — 2 again  and  U\  - 1/2 ,w2  — 1/3. 


Proceeding  to  problem  P'l  A,  we  get  the  < 

1:  maximize  y0 
subject  to  >0<>i+(1/3)>'2+1, 
*,<  (l/2)>,+(l/3).y  2+1/2, 
^i+^z^l. 
y{+y  j>2, 

(l/3)y2>0, 

ytr. 

Solutions:  >■,  - 2,^2  - 0,  yt  - 3/2. 

2:  maximize  y„ 
subject  to  y„Kyx+{\/3)y2+\. 
yMUDy  ,+>-2+1/2, 
3'i+>’2^1, 
yt+y2>2, 

(l/3)y2  - 0, 


maximize  ya 
subject  to>„<>,+l, 
y„£  (l/2)yx+(l/i)y2+l/2, 
>i+>2>1. 
yx+y2>2, 

(l/3)>2  - 0, 

>-€K 

Solutions:  yx  - \,y2  - 0,  y„  - 3/2. 
maximize  y„ 
subject  to  j'o^^i+J'z+l. 
>0<(l/2)y,+(l/3)^2+l/2, 
yx+y2>i, 
yt+y2>2, 

(l/3)y2>0, 

ycr. 
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Solutions:  y{  - 2 ,y0  - 2,y2  - 0,  y„  = 3/2. 
3:  maximize  y„ 
subject  to  >><, <.v  i+(l/3 )> 2+1. 
y„<(\/2)yi-l2/3)y  2+5/2, 

y\+y2>h 

>l+3'2<2, 

(l/3)^2<0, 

>€  Y. 

Solutions:  - 1 ,>>2  “ 0,  y0  - 2. 

4:  maximize  y0 
subject  to  >'0<>’i+(1/3)>'2+1, 
>0<-(l/2)^,+5/2, 

•Vl+j^l, 

>',+>'2<2, 

(l/3)>2  - 0, 
ytY. 

Solutions:  yi  — \.y2  " 0.  y0  =»  2. 


Solutions:  y,  = 0,  y2  = 2,ya  - 5/2. 
maximize  y0 
subject  to  j'0^y|+>'2+l, 
y0^-(\/2)y,-(2/3)y2+5/2, 
y\+yi>\, 
y 1+-V2^  2, 

(1/3^2  “0, 
y£Y. 

Solutions:  = 1,>2  = 0-  y0  “ 2. 

maximize  y0 
subject  to  j'0<>'i+j'2+l» 
j'0<-(l/2)>'1+5/2, 

>’l+>’2^1. 

>'l+>’2<2, 

(l/3)y2>0, 

y*Y 

Solutions:  y,  - 0,  y2  « 2 ,y0  - 5/2. 


The  best  solution  is  y\  - 0,  y2~  2,  and  y0  - 5/2,  which  gives  UB  - 5/2.  When  the 
solution  y\  “ 0,  y2  — 2 is  used  in  problem  P'1,  we  get  LB  — 5/2,  and  thus  the  optimal  solution 
appears  to  be  the  extreme  point  corresponding  to  the  largest  value  of  z. 

In  the  following  section  we  consider  another  class  of  quasi-concave  function  for  which  the 
GBD  procedure  is  applicable. 

Bl-NONLINEAR  PROGRAMMING 

A second  type  of  quasiconcave  function  amenable  to  the  proposed  technique  is  a bi- 
nonlinear  function.  The  proposed  problem  is  of  the  following  kind: 

m 

P:  maximize  z - £/,(.y)  ' c,(y ) 

1-1 

subject  to  y€Y.  The  functions  ft(y)  are  assumed  to  be  concave,  and  the  functions  c,{y) 
to  be  linear.  We  also  assume  that  f,(y)  ^0  and  ct(y)  >0  for  y€  Y,  /— 1 m [5],  The  prob- 

lem P'  is  formed  as  follows: 


P’:  maximize  2 - £x, 

/-I 

subject  to  fj(y)-x,ICi(y)  >0,  /-l,  ....  m,  y€Y,  x€X. 

The  set  X is  determined  exactly  as  it  was  previously.  Note  that  this  problem  also  satisfies  the 
assumptions  for  solution  by  the  GBD  procedure.  Once  again,  for  fixed  y problem  P'  has  a 
closed  form  solution  as  m linear  programming  problems,  so  that  Kuhn-Tucker  multipliers  u,  are 
again  guaranteed  to  exist. 

For  a given  u vector,  problem  P'l  A can  be  formulated  as  follows: 

P'lA:  maximize  y„ 
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t 

m m 

subject  toy„<  £ uifAy ) + £ maximum  (x,[l-w,/c,( >>)]), 

— I (-1 

yt  Y. 

Here,  once  again,  the  solution  in  terms  of  x,  depends  on  the  sign  of  the  terms  l-K,/e,(>>).  If 
ui/c,(y)  ^ 0,  we  have  x,  — 0,  and  if  \—uJ ci(y)'^0,  we  have  x,  — x,“.  Thus,  the  solution 
procedure  for  this  problem  exactly  parallels  the  procedure  for  the  nonlinear  fractional  problem. 
It  should  be  pointed  out,  however,  that  problem  P'lA  is  not  a sequence  of  linear  programs, 
since  it  contains  the  terms  «,/c,(y)  for  problems  with  x,  ™ x".  A saving  grace  may  be  that 
since  the  functions  c,(y)  are  linear,  a simple  change  of  variables  can  transform  these  terms  into 
separable  nonlinear  functions  for  which  linear-programming  approximations  are  available. 
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ABSTRACT 

Consider  a single-server  exponential  queueing  loss  system  in  which  the  ar- 
rival and  service  rates  alternate  between  the  pairs  (A, . *»|)  and  (\2.  Mi>.  spend- 
ing an  exponential  amount  of  time  with  rate  cot,  in  (A(,  /a,),  / — 1.2.  It  is 
shown  that  if  all  arrivals  finding  the  server  busy  are  lost,  then  the  percentage  of 
arrivals  lost  is  a decreasing  function  of  c.  This  is  in  line  with  a general  conjec- 
ture of  Ross  to  the  eflfect  that  the  "more  nonstationary*  a Poisson  arrival  pro- 
cess is,  the  greater  the  average  customer  delay  (in  infinite  capacity  models)  or 
the  greater  the  percentage  of  lost  customers  (in  finite  capacity  models).  We 
also  study  the  limiting  cases  when  C approaches  0 or  infinity. 


1.  INTRODUCTION 

This  paper  is  a continuation  of  a study  of  queueing  models  with  nonstationary  Poisson 
arrivals  begun  in  Ref.  [2],  where  it  was  conjectured,  and  verified  in  a special  case,  that  a queue- 
ing system  with  nonstationary  Poisson  arrivals  will  lead  to  larger  average  customer  delays  than 
would  a similar  model  having  stationary  Poisson  arrivals  with  the  same  average  arrival  rate.  In 
order  to  investigate  this  conjecture  further  we  consider  a single-server  loss  system  that  oscillates 
between  two  feasible  levels  denoted  by  1 and  2.  When  the  system  is  at  level  / (/  — 1,2),  the 
arrival  process  is  a Poisson  process  with  rate  A,  and  the  service  times  are  exponential  random 
variables  with  rate  fi,.  The  time  interval  during  which  the  system  functions  at  level  / is  also  an 
exponential  random  variable  with  rate  ca,  where  c is  a constant,  i.e.  the  persistence  of  the  sys- 
tem at  any  level  is  governed  by  a random  mechanism:  if  the  system  is  functioning  at  level  >,  it 
tends  to  "jump”  to  the  alternative  level  with  Poisson  rate  ca,. 


‘This  research  has  been  partially  supported  by  the  Office  of  Naval  Research  under  Contract  N00014-77-C-0299  and  the 
Air  Porce  Office  of  Scientific  Research,  AFSC,  USAF,  under  Orant  AFOSR-77-321J  with  the  University  of  California. 
Reproduction  in  whole  or  in  part  is  permitted  for  any  purpose  of  the  United  States  Government. 

483 

/ 


484 


S.  FOND  & S.  M.  ROSS 


We  suppose  that  an  arriving  customer  will  only  enter  the  system  if  the  server  is  free  when 
he  arrives.  Let  L(c)  denote  the  proportion  of  customers  that  are  lost  to  the  system.  In  the 
following  section  we  show  that 

He)  is  decreasing  and  convex  in  c. 

It  should  be  noted  that  the  (time)  average  arrival  and  service  rates,  X and  JL,  are  given  by 

- X|<*2  + X^i  _ Mi«2  + M2«l 

X — , M “ 

t*[  + a2  <*|  + a2 

and  are  thus  independent  of  c.  The  purpose  of  the  constant  c is  to  regulate  how  fast  the  system 
changes  levels;  thus,  the  larger  c is,  in  some  sense  "the  more  stationary  the  process  is."  Indeed, 
as  c approaches  infinity,  the  system  converges  to  a stationary  one. 

2.  THE  LOSS  FUNCTION  L(c) 


The  system  can  be  analyzed  as  a continuous-time  Markov  process  with  states 
[(m,i)\m  =0,1  and  i = 1,2),  where  m denotes  the  number  of  customers  in  the  system  and  / 
denotes  the  level  of  the  system.  The  transition  probabilities  are  stationary  and  satisfy  the  for- 
ward Kolmogorov  differential  equations.  Moreover,  for  all  (/»,/'),  the  limiting  probabilities,  call 
them  Pm„  exist  and  are  independent  of  the  initial  state.  The  set  [Pmi]  satisfies  the  following 
balance  equations: 


(la) 

(X| 

+ 

c<*\)P oi  “ P[P\\  + 

ca2P  02 

(lb) 

(Ml 

+ 

cat\)P ii  “ X)Poi  + 

ca2p  12 

(2a) 

(X2 

+ 

ca 2)P02  “ P2p  12  + 

caxP  oi 

(2b) 

0*2 

+ 

Ca2)P  12  “ X2P 02  + 

ca,P,, 

with 

(3) 

P 01 

+ ^*11  + ^*02  + ^*12 

= 1. 

Let  He)  denote  the  proportion  of  customers  lost  to  the  system.  Since 

XL(c)  = K\P  u + X2P  i2, 

we  can  calculate  He)  by  finding  Pn  and  Pn.  Before  doing  that,  let  us  note  that  the  proportion 
of  time  the  system  is  in  level  1 is 


(4) 


Poi  + Pn 


a2 

“I  + “2 


which  can  be  obtained  either  by  adding  (la)  and  (lb)  together  and  substituting  (3),  or  by  con- 
sidering the  system  as  an  alternating  renewal  process.  Similarly, 


(5) 


The  easiest  way  to  solve  for  Pn  and  Pu  is  to  put  (la)  and  (lb)  in  a matrix  form  as  fol- 
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Similarly,  for  (2a)  and  (2b): 

(A24ca2)  P(,2  c«i^oi 

^ _*2  (/i2  + ra2)  P 12  ” cot , • 

Putting  (6)  and  (7)  together  yields 


’)  “M2 

P 02 

c2a , a 2Po2 

<0i2  4 fO]) 

Pn  “ 

c2a  ,02^12; 

f(  (r>|p24u2  p 

,)  +m2(a. 

+ M|)IP|2- 

From  the  first  row  of  (8)  we  obtain 

[c(<*|A2  + a2Al)  + A2(A|  +M|)IPo2  “ fr(o|ii24u20il)4p2(A!40t|)]/J 

Therefore, 

P |2  f ( « | A , + «t  iA  , ) + A ,t  A | 4 p ( 1 
P n?  r(a,Mj  + «j  Mi* -*-M2<A| +M|) 

Hence,  by  (5)  and  (9), 

p a l »tu  ,A;  ♦ a^A  |)  ♦ Ait  A ! + 

12  U|  + Oj  I U|(A)  + »ijl  ♦«•(*  »p  1 1 ♦ <A  < d|(U,  + /i,l 


a | ilu^  «|i  *»  li  • p I 

02  II  | 4 (1  I |«  I A ♦ p ♦ It  1 1 » I A * n It  A » N I 

Due  to  the  symmetry  of  the  equation*  ..  . t .»*  • in* 

«i  rl«  * • • 4 • » t ♦ » I 


a j 

Pn-— rr 


rii  + oj  i |u  li  > n • » ll  • m " * * m yi 

m2  '« * • * * 


ri«  I A * p J * m t 


• Mi  j 


Thus,  we  have 

At(c)  - 
Differentiation  yields 
AL  (c)  - 


• 4 - - 4 « * > • a . ta  • «i ,» 

I a | 4 it  j)  (i  |m  ( A a p d ♦ .«  '4  'a  • ^ Mi  ♦ ^ i| 

to  | -f  M (<|m  ,(  A,  ♦ ii,|  ♦ ( 4 4 ft  »|  a <A  ♦ |4  I f A , a 04  | ■ 


KL"U)  - 2a|02(A^42-  A20i|)tt«i|(A2a0i>)  apjfA^p,)! 

(or  | + a 2)  {<'  |«  |(  A j 4 0i  j)  4 a2<  A , 4 01  ()|  4 < A , 4 p ,)( A j 4 p }) ) ' 
There  are  two  cases  to  consider. 


CASE  1:  A|/i2  - A2 01,  - 0;  i.e.,  the  traffic  intensities  A|/0i,  and  A >/p2  are  equal,  say  to  p. 

In  this  case  L'(c)  — 0,  and  thus  L(c)  is  independent  of  the  value  c.  Moreover,  we  have 
simple  solutions  for  the  Pm's  in  this  case,  namely 
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“2 

1 

a 

1 + «2 

1 + p 

«2 

P 

a 

1 + “2 

1 + p 

a 1 

1 

a 

1 + “2 

1 + p 

«! 

P 

+ ar2 

1 + p ' 

Hence,  Plf  the  proportion  of  time  the  system  is  busy,  is 

p,  = p„  + p12-  , , 

l + p 

and  Po,  the  proportion  of  time  the  system  is  empty,  is 

Pq  = P0\  + ^02  “ J + p • 

In  terms  of  P0  and  P|t  the  system  functions  as  an  ordinary  M/M/1  loss  system  with  traffic 
intensity  p.  The  loss  function  is  found  to  be 

He)  - -r^r — • 


CASE  2:  — XjPi^O. 

In  this  case  L'(.c)  < 0 and  L"(c)  > 0.  Hence,  He)  is  a decreasing  convex  function  of 
the  value  c. 

Therefore,  if  the  ratio  of  the  time  the  system  stays  at  each  level  is  fixed,  then  the  faster 
the  system  alternates  between  these  two  levels,  the  better  the  system  is  (in  terms  of  the  loss 
function). 


3.  EXTREME  CASES 


We  have  shown  that  He)  is  a strictly  decreasing  function  of  c when  the  traffic  intensities 
X,/p,  and  X2/M2  arc  not  equal.  Now  let  us  study  the  two  extreme  cases:  (a)  c — «*»,  i.e.  the 
system  alternates  extremely  fast  between  level  1 and  level  2 or,  equivalently,  the  mean  time 
that  the  system  stays  at  each  level  approaches  0;  (b)  c — * 0,  i.e.  the  system  alternates  extremely 
slowly  between  level  1 and  level  2 or,  equivalently,  the  mean  time  that  the  system  stays  at  each 
level  is  becoming  infinitely  large. 


CASE  1:  c—  00 

X lim  L(c) 


(or  [X  2 + a2X|)2 

(a,  + a2)lai^2  + ^2)  + “2^1  + M|)l 


implying  that 
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Furthermore,  the  proportion  of  time  that  the  system  is  busy  can  be  obtained  by 


P i - lim  + lim  Pl2  - — — 

C —OO  c — •«»  /X  "4"  X 


and  the  proportion  of  time  that  the  system  is  idle  is 


P0  - lim  P0 , + lim  P 02  - — — - 

M + X 


Thus,  the  limiting  system  is  equivalent  to  a no-queue-allowed  M/M/l  system  with  constant 
arrival  rate  X and  service  rate  JL. 


Since  L(c ) is  decreasing,  the  value  — is  the  smallest  value  the  system  can  achieve 

M + X 

for  the  loss  function. 


CASE  2:  c — 0 


X lim  L (c) 


Kia2(,K2  + m2)  + XfajU]  + Mi) 


(<»i  + a2)(X|  -I-  mi)U2  + M2) 


«2 


, 2 


M 


'2 


“l  + "2  *I+M|  “|  + a2  X2  + M2’ 

and  the  proportion  of  time  that  the  system  is  busy  is 

<*•>  X 1 


P\  - lim  Pu  + lim  Pn  - —=1 ^ 

<• -O  i—o  or | + a2  /a ] + X 1 a|  + a2  fi2  + k2 


The  proportion  of  time  the  system  is  idle  is 


P0  - lim  P0l  + lim  P02  - — — + — — — — 

r— 0 <—o  at | + a2  Mi  + Xj  at|  + a2  fi2  + \2 


Thus,  the  limiting  system  functions  as  the  (time)  average  of  two  independent  M/M/ 1 loss  sys- 
tems, one  with  arrival  rate  X,  and  service  rate  Mi.  and  the  other  with  arrival  rate  X2  and  service 
rate  m2- 


4.  RIGHT  AND  WRONG  ARRANGEMENTS 


Let  us  assume  X!  < X2  and  mi  < M2,  and  compare  the  system  R with  levels  (X,,mi), 
(X2,m2)  to  the  system  W with  levels  (X),M2)i  (X2. m 1)  under  the  condition  aj  - a2.  In  other 
words,  the  system  R has  the  arrangement  such  that  the  server  with  slow  service  rate  goes  on 
the  shift  with  the  slow  arrival  rate  and  the  person  with  the  fast  service  rate  goes  on  the  shift 
with  the  fast  arrival  rate.  The  system  W is  arranged  the  other  way  around.  If  we  denote  the 
loss  functions  of  the  systems  R and  W by  LK  and  Lw , respectively,  then  a simple  algebraic 
computation  yields  that  LR(c)  < Lw(c ),  and  so  the  system  R is  better  than  the  system  W in 
the  sense  of  loss  function. 


5.  FINAL  REMARKS 


The.  model  considered  here  is  similar  to  ones  considered  in  Refs.  [1]  and  [3].  However, 
the  results  obtained  in  these  papers,  being  in  terms  of  the  root  of  some  polynomial,  do  not 
seem  to  enable  one  to  draw  the  type  of  conclusion  obtained  in  the  present  paper. 


' • '*■ 
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ABSTRACT 

A transportation  system  has  N vehicles  with  no  capacity  constraint  which 
take  passengers  from  a depot  to  various  destinations  and  return  to  the  depot. 
The  trip  times  are  considered  to  be  independent  and  identically  distributed  ran- 
dom variables.  The  dispatch  strategy  at  the  depot  is  to  dispatch  immediately, 
or  to  hold  any  returning  vehicles  with  the  objective  of  minimizing  the  average 
wait  per  passenger  at  the  depot,  if  passengers  arrive  at  a uniform  rate. 

Optimal  control  strategies  and  resulting  waits  are  determined  in  the  special 
case  of  exponentially  distributed  trip  time  for  various  N up  to  /V  - 15.  For 
N » 1,  the  nature  of  the  solution  is  always  to  keep  a reservoir  of  vehicles  in 
the  depot,  and  to  decrease  (increase)  the  time  headway  between  dispatches  as 
the  size  of  the  reservoir  gets  larger  (smaller).  For  sufficiently  large  N,  one  can 
approximate  the  number  of  vehicles  in  the  reservoir  by  a continuum  and  obtain 
analytic  expressions  for  the  optimal  dispatch  rate  as  a function  of  the  number 
of  vehicles  in  the  reservoir.  For  the  optimal  strategy,  it  is  shown  that  the  aver- 
age number  of  vehicles  in  the  depot  is  of  order  Ar  . These  limit  properties  are 
expected  to  be  quite  insensitive  to  the  actual  trip  time  distribution,  but  the  con- 
vergence of  the  exact  properties  to  the  continuum  approximation  as  N — °°  is 
very  slow. 


1.  INTRODUCTION 

We  are  concerned  here  with  strategies  for  dispatching  vehicles  from  a depot  of  a public- 
transportation  system.  The  system  consists  of  N vehicles,  each  of  which  takes  passengers  from 
the  depot  to  various  destinations,  and  then  returns  after  some  random  rip  time  to  make 
another  trip.  Passengers  arrive  at  the  depot  at  a constant  rate,  and  vehicles  r -ave  sufficient  capa- 
city that  a passenger  can  always  board  the  next  departing  vehicle.  The  objeciive  is  to  minimize 
the  long-time  average  wait  per  passenger  at  the  depot. 

A similar  problem  was  previously  considered  by  Osuna  and  Newell  [4],  with  emphasis  on 
the  case  of  small  /V,  particularly  A — 1 or  2.  For  A'  — 2,  the  coefficient  of  variation  of  the  trip 

time,  C(T)  - Var2  ( T)/E(T ),  was  further  assumed  to  be  small  compared  to  1.  The  following 
analysis  will  approach  the  problem  from  the  opposite  extremes,  cases  of  AT  » 1 and/or  trip 
times  with  large  variances,  where  vehicles  are  likely  to  pass  each  other  enroute. 


•This  research  was  supported  in  part  by  the  National  Science  Foundation  under  grant  MPD  72-05068A03. 
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It  was  shown  by  Osuna  and  Newell  that  if  the  dispatch  times  r,  < t2  < ...  define  a 
sequence  of  headways 

H,  — t , — Tj_|,  / — 2,  3,  .... 

which  are  identically  distributed  random  variables  satisfying  a law  of  large  numbers 

-If/,  - E(H), 

n ,-i 

and  if  passengers  arrive  at  a uniform  rate  independent  of  dispatch  times,  the  average  wait  per 
passenger  is 

(1.1)  £(  W)  - E(H*)/2E(H)  - (\/2)E(H)[\  + CHH)]. 

It  is  therefore  desirable  to  keep  both  the  mean  headway  and  the  variance  of  H small. 

We  assume  here  that  the  stochastic  properties  of  the  trip  times  are  given  and  that  the  only 
mechanism  for  control  is  to  delay  a dispatch.  The  minimum  value  of  E(H ) is  achieved  by 
dispatching  vehicles  with  no  delay,  but  then  the  random  trip  times  generally  lead  to  a large 
value  of  C2(.H).  The  optimal  control  strategy  involves  an  increase  of  the  "effective  trip  time" 
in  order  to  improve  the  regularity.  For  small  N,  this  strategy  typically  requires  that  one  merely 
delay  a vehicle’s  departure  if  it  returns  too  early,  but  there  will  seldom  be  more  than  one  vehi- 
cle at  the  depot.  For  N » 1,  however,  the  optimal  strategy  will  typically  involve  maintaining 
a reservoir  of  vehicles  at  the  depot,  so  that  one  will  almost  always  have  a vehicle  to  dispatch  at 
some  desired  dispatch  time.  The  actual  number  of  vehicles  in  the  reservoir  will  fluctuate  and 
may  occasionally  vanish.  The  optimal  strategy  describes,  among  other  things,  what  average 
fraction  of  the  total  number  of  vehicles  one  should  keep  in  the  reservoir  in  order  to  stabilize 
the  departures. 

Our  analysis  of  the  problem  is  in  two  parts.  First  we  give  an  exact  formulation  of  the 
problem  for  exponentially  distributed  trip  times.  An  analytic  solution,  although  conceptually 
simple,  becomes  unmanageable  if  N becomes  large  (even  N - 4),  but  we  do  show  numerical 
solutions  for  \ 15.  In  the  second  part  we  derive  an  approximate  forumlation  for  the  prob- 

lem, the  solution  of  which  gives  the  asymptotic  properties  for  N — > °°.  The  numerical  and  the 
asymptotic  solutions  agree  fairly  well  for  N - 15.  For  N » 1,  the  optimal  expected  number 
of  vehicles  kept  at  the  depot  is  shown  to  be  proportional  to  A 1/3 

Although  the  postulate  of  an  exponentially  distributed  trip  time  is  not  very  realistic  for 
most  real  transportation  systems  (which  typically  have  a coefficient  of  variation  in  trip  time  con- 
siderably less  than  1),  the  results  obtained  here  should  give  a crude  upper  bound  on  the 
optimal  number  of  vehicles  kept  at  the  depot  for  systems  with  smaller  variation  in  trip  time. 
For  sufficiently  large  Af,  however,  one  can  argue  that  the  process  of  returning  vehicles  should 
be  approximately  a Poisson  process,  regardless  of  the  detailed  nature  of  the  trip  time  distribu- 
tion or  the  dispatch  strategy.  The  optimal  control  strategy  should,  therefore,  be  insensitive  to 
the  trip  time  distribution  provided  that  the  uncertainty  in  trip  time  is  large  compared  with  the 
mean  headway  (so  that  a typical  vehicle  passes  or  is  passed  by  many  others  before  it  returns). 
Unfortunately  the  convergence  to  such  an  asymptotic  behavior  is  very  slow  and  of  questionable 
accuracy  for  any  reasonable  values  of  N and  C(T)  encountered  in  real  systems. 

2.  Formulation 

Let  T,  be  the  trip  time  of  the  vehicle  which  leaves  the  depot  at  time  r„  / - 1,  2 

The  T,'s  are  assumed  to  be  independent  identically  distributed  random  variables  with  a continu- 
ous distribution  function  Fr(z)  - P(T,  < z).  We  can  measure  time  in  any  arbitrary  units. 
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Hereafter,  all  times  will  be  interpreted  to  be  measured  in  units  of  E(T)/N  (the  average  head- 
way with  no  control).  In  these  units 

(2.1)  E(T)N-l. 

At  any  time,  the  information  that  is  relevant  to  the  future  behavior  of  the  system  is  the 
number  of  vehicles,  x,  in  the  depot;  the  number  of  passengers,  M , waiting  to  be  served;  and 
the  times  f|  ^ i2  > ...  > /*_*  since  the  last  dispatches  of  all  the  vehicles  enroute.  Since  there 
is  nothing  to  be  gained  by  dispatching  more  than  one  vehicle  at  a time,  the  t's  will  satisfy 

/,  > h > •••  >tN-x. 

We  assume  that  at  any  time,  we  know  the  values  of  x,  r,,  ....  tN_x,  but  we  cannot 
observe  M.  Since  passengers  arrive  at  a uniform  rate,  we  will  base  our  strategy  on  the  expected 
value  of  A/,  E(M),  which  is  proportional  to  the  time  / since  the  last  dispatch.  (The  time  / since 
the  last  dispatch  may  be  equal  to  one  of  the  /,  or  it  might  be  a time  since  the  previous  dispatch 
of  one  of  the  x vehicles  in  the  depot.)  Since  the  optimal  strategy  does  not  depend  upon  the 
arrival  rate  of  passengers,  we  arbitrarily  take  this  rate  to  be  1. 

We  can  define  a state  of  the  system  as  the  vector  <x,t,  /, /\-,>.  If  at  all  times  we 

follow  a strategy  (dispatch  or  not)  which  depends  only  upon  the  present  state  of  the  system,  the 
future  behavior  will  depend  only  upon  the  present  state,  and  hence  we  have  a Markov  process. 

Generally,  this  Markov  process  is  quite  complicated.  If,  however,  we  assume  that  the  trip 
times  of  the  vehicles  are  exponentially  distributed,  then  the  future  arrival  times  of  the  vehicles 
at  the  depot  are  independent  of  previous  dispatch  times.  The  future  behavior  of  the  system 
depends  only  on  the  state  vector  <x,  />.  This  state  space,  as  shown  in  Figure  1,  consists  of 
W+l  real  lines,  / ^ 0,  x = 0,  1 , . . . ,N. 


Since  the  dispatch  strategy  depends  only  on  the  state  of  the  system,  it  is  specified  by  a set 
of  points  in  the  state  space  at  which  one  should  dispatch  a vehicle,  and  a complement  set  of 
points  at  which  one  should  hold  every  vehicle.  It  is  intuitively  obvious  that  if  we  should 
dispatch  whenever  the  state  is  <x,  />,  then  we  should  also  dispatch  whenever  the  state  is 
<x,  /'>,  for  /'  > /.  Therefore,  we  can  assign  numbers  ax  for  each  x such  that  we  hold  every 
vehicle  if  we  are  at  any  point  <x.  />,  / < ax,  and  we  dispatch  one  if  we  are  at  a point 


Figure  I — The  stale  space.  The  light  solid  lines  (0,  a,) 
are  the  set  of  hold  points,  the  dark  solid  lines  (a,,  a,.,  1 are 
the  set  of  dispatch  points,  and  the  light  dashed  lines 
(a,_/,  oo>  are  the  set  of  unreachable  points.  Two  typical 
state  trajectories  ABCEDFG  and  A’  B’  C‘  D'  E’  F’  G’  are 
drawn  with  dark  solid  lines  and  dark  dashed  lines,  respec- 
tively. 
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<x,  t>,  t > ax.  Note  that  a0  - °°  is  the  only  possible  value  for  aq.  It  is  also  obvious  that  if 
we  should  dispatch  at  a point  <x,  />,  then  we  should  dispatch  at  a point  <x+l,  t>  because 
we  can  better  afford  to  use  the  vehicles  if  we  have  more  of  them.  It  follows  that 

(2.2)  °o  - a0  > a,  ^ a2  > • • • ^ 0/v  > 0. 

Figure  1 also  shows  two  types  of  state  trajectories.  As  long  as  no  departure  occurs,  the 
state  moves  at  unit  speed  from  left  to  right  with  upward  jumps  of  magnitude  one  at  each  arrival 
of  a vehicle  (as  at  BC,  DE,  B’C’,  — ).  At  any  departure  time,  however,  there  is  a downward 
step  by  one  and  a return  to  / - 0 (as  in  GH  or  F’G’).  A dispatch  will  occur  when  the  state 
<x,  t>  first  reaches  a point  with  t ^ ax.  This  may  occur  either  because  a vehicle  arrived  at 
time  t and  a,  < t < a,_|(  as  at  point  G,  or  the  state  moved  to  the  point  <x,  ax>  continu- 
ously as  at  F'.  It  is  not  possible  to  reach  states  <x,  t>  with  t > ax^  shown  by  the  broken 
lines;  dispatches  occur  only  in  the  intervals  [ax,  ax_ ,]. 

The  problem  now  is  to  find  that  strategy,  i.e.,  a sequence  a',  . , a’N  among  all  the  possi- 
ble a,  > a 2 > ...  > aN,  such  as  to  minimize  the  expectation  of  wait  of  a randomly  chosen 
passenger.  Under  a strategy  of  no  control,  a vehicle  is  dispatched  as  soon  as  it  returns  no 
matter  how  short  the  headway,  i.e.,  — . . . — aN  — 0,  a0  — °°.  The  only  states  which  can  be 

reached,  however,  are  those  with  x - 1 or  0. 

For  any  strategy  la,},  the  states  of  the  system  <x,  0>,  x — 0,  1,  ....  N — 1,  immedi- 
ately after  a dispatch  define  a finite-state  Markov  process.  If  qxv  is  the  conditional  probability 
of  having  y vehicles  after  the  kth  dispatch,  given  that  we  had  x vehicles  after  the  (Ar-l)th 
dispatch,  and  px(k)  is  the  probability  of  having  x vehicles  after  the  kth  dispatch,  then  the  px(.k) 
satisfy  the  equations 

(2.3)  Pv(*+ 1)  - V qxy  px(.k),  y-  0 \ - 1. 

JC-0 

After  sufficient  time  (Ac  — <»),  the  pv(k)  will  approach  an  equilibrium  distribution  pv  which 
satisfies  the  equations 

N- 1 N- 1 

(2.4)  p - I q p I px  - 1. 

' Jt-0  y *-o 

To  evaluate  the  qx  y,  we  note  that  the  assumption  of  exponentially  distributed  trip  times 
means  that  (in  the  units  (2.1)) 

(2.5)  Ft(z)  - 1 - exp(-z/A0. 

If  there  are  x vehicles  at  the  depot,  and  no  dispatch  occurs  during  a time  interval  a,  then,  for 
each  of  the  N - x vehicles  enroute,  there  is  a probability  1 - exp(.-a/N)  that  it  will  return  in 
the  time  interval  a,  and  txp(-a/N ) that  it  will  not.  The  probability  of  exactly  /arrivals  in  an 
interval  a,  given  that  x vehicles  were  at  the  depot  at  the  beginning  of  the  interval  and  no 
dispatches  occurred  during  the  interval,  is 

[exp(-a//V)]/v-,'-'|l  -exp(-a//V)K  (2.6) 

a binomial  distribution  with  / successes,  N - x - I failures,  and  probability  of  failure 
exp(-<i/A/). 
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From  Figure  1,  one  can  readily  see  that  a transition  from  the  state  <x  0 > to  <a,  0>, 
with  0 < o-  ^ y — 1,  will  occur  if  and  only  if  at  most  y — x vehicles  arrive  in  the  time  interval 
(0,  ay).  The  probability  of  this  event  is,  therefore, 


(2.7)  Qx  = 


y- 1 

I 

»rM) 


v- x 

I 

<r— 0 


N-x 

<j 


[exp(-a,/A,)]*-jr-"  (I  - exp(-fl,/A')]" 


~ FB(y  — x\  N — x,  1 - exp (-aJN)) 

for  x - 1 < y A'  - l,  x - 0 N- 1 

“ 0 for  y < x - 1, 


in  which  FB(x\  n,  p)  is  the  cumulative  binomial  probability  distribution  [5],  and 

(2.8)  dx.v  = O.t.v  + 1 ~ Qx.y 


The  solution  of  the  system  of  TV  linear  equations  (2.4),  with  qxv  given  by  (2.7)  and  (2.8), 
determines  the  px  as  functions  of  the  strategy  {av|.  One  can  also  determine  the  stationary 
headway  distribution 

(2.9)  F(h)  = P(H  > h) 

in  terms  of  the  px  and  ax. 

For  h < aN,  F(h)  = 1,  whereas  for  ar+1  < h < ax,  x = 0,  1 2V  — 1,  it  is  given  by 

— X 

(2.10)  F(h)  - I pp  P(H  > /j|/3  vehicles  present  at  the  start  of  H) 

0-0 

Jf 

" Pp  P(  at  most  x-p  arrivals  in  (0,  h)  |/3) 

X 

“ J-Q  Pp  Fb(x  - P\  N - p,  1 - exp(-/t/jV)). 

From  this,  one  can  evaluate 

(211)  E(H)  - C F(h)  dH,  E(H *)  - 2 f"  h F(h)  dh. 

•'O  •'O 

and 

E(W)  - £(W2)/2£(/y) 

as  functions  of  the  (a,}.  The  problem  is  thus  reduced  to  minimizing  £(  W)  with  respect  to  the 
ax,  with  > a2  >,  ...  ~&aN. 

Except  for  small  N,  the  evaluation  of  E(W)  and  ax  is  extremely  tedious,  but  one  can 
obtain  some  bounds  on  E(W)  and  the  a’.  We  first  note  that,  if  we  dispatch  with  no  control, 
the  arrivals  of  vehicles  form  a Poisson  process  with  C(H)  - 1 and,  from  (1.1), 
£(  W)  - E(H)  - E(T)/N  - 1.  Therefore 

E*(W)  = E(W\a\ a'N)  < 1. 


On  the  other  hand,  from  (1.1),  we  have 


E(W) 


E(H) 


[1  + C7(H) ) > 


E(H) 

2 


2 
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One  can  show  that  the  minimum  value  of  E(H)  is  achieved  with  no  control,  i.e.,  £(//)  > 
E(T/N)  - 1.  Consequently, 

(2.12)  1/2  < £*(FF)  < 1. 

The  lower  bound  would  obtain  if  one  could  maintain  regular  headways  with  negligible  control 
delays.  The  goal  for  any  finite  N is  to  make  £*(  1+0  as  close  to  1/2  as  possible. 


Although  E(W)  is  a complex  function  of  a,,  ...  , aN_ |t  its  dependence  upon  a » is  rela- 
tively simple.  For  any  fixed  values  of  a, fl/v-i,  one  can  show  from  (2.10),  (2.11)  that 

6 ft//2)  _ dE(H) 

baN  2 ° N daN 

and  that  the  equation 


dE(,W)/da„  = 0 

is  satisfied  provided  that  aN  satisfies  the  equation 
(2.13)  a„=E(fV). 


Thus,  for  any  choice  of  a, 0/v-i,  the  optimal  aN  must  satisfy  (2.13).  This  does  not 

give  aN  "explicitly",  because  E(W)  depends  upon  aN>  but  E(W)  varies  so  slowly  with  aN  that  a 
sequence  of  successive  approximations  will  converge  very  rapidly. 

Equation  (2.13)  can  also  be  derived  by  a dynamic-programming  argument,  such  as 
described  in  reference  [4],  for  the  special  case  N = 1. 

Since  we  know  that  £*(  W)  > y,  it  follows  that 

(214)  y < E*(W)  - a'N  < a'N.x  < ...  < a0*  - ». 

Also,  the  strategy  aN  - fly-i  ” • • • “ «i  “ y will  give  a smaller  E(W)  than  no  control, 
aN  - a/v_i  - ...  — a,  - 0,  therefore  a better  upper  bound  for  £*( W)  than  (2.12). 


3.  NUMERICAL  RESULTS  FOR  SMALL  N 

For  N - 1 and  2 ( and  perhaps  3),  evaluation  of  the  optimal  control  can  be  done  analyti- 
cally, but  for  larger  N the  solution  of  (2.4)  and  the  minimization  of  E(W)  with  respect  to  the 
ar  must  be  done  numerically. 


The  solution  for  N - 1 was  described  in  reference  [4)  even  for  a general  trip-time  distri- 
bution. The  optimal  strategy  is  to  dispatch  the  single  vehicle  immediately  if  it  arrives  after 
some  time  a,,  but  dispatch  at  time  a!  if  it  arrives  before  time  a,.  Equation  (2.4)  has  the  trivial 
solution  p0  “ 1-  The  headway  distribution  (2.10)  is  a truncated  exponential 


£(/») 


1, 

exp(-/i), 


0 < h < a,, 
a,  < h, 


giving 


£(//)  - a,  + exp(-a,),  and  £(//2)  - a,2  + 2(a,  + l)exp(-a,). 


OPTIMAL  DISPATCHING  STRATEGIES 


495 


{ 


The  optimal  a,  is  given  by  (2.13) 

. fl,'2/2  + (a\  + l)exp(— a,*’ 

(3.1)  a, r— —7 — r . 

a,  + exp(-a,) 

which  can  be  solved  by  successive  approximations.  If  we  substitute  any  y'th  approximation 
a,(y)  in  the  right-hand  side  of  (3.1)  and  evaluate  the  (y  + l)th  approximation  as  the  value  of 
the  left-hand  side,  this  sequence  of  approximations  will  converge  very  rapidly.  For  example,  if 
we  take  a trial  value  of  a,'(0)  - 1/2,  then  a ,* ( 1 ) - 0.935  and  al"(2)  - 0.901  (already  correct  to 
three  decimal  places). 

As  compared  with  no  control,  the  optimal  control  has  reduced  E(W)  from  1 to  approxi- 
mately 0.90.  This  reduction  is  achieved  through  increasing  E(H)  from  1 to  1.307  and  decreas- 
ing CHH)  from  1 to  0.38.  This  illustrates  how  the  average  wait  can  be  reduced  through  an 
improvement  in  the  regularity  in  service  at  the  expense  of  longer  average  headways. 

For  /V  - 2,  there  are  two  control  parameters  at  and  a2.  The  transition  probabilities  (2.7) 
become 

(?o.  i ” « o.o  “ 1 “ <7o.  i “ 2exp(— a ,/2)  - exp(-a ,, 
and 


and  the  solution  of  (2.4)  is 

Po  - 1 ~ P\  - 
which  is  independent  of  a2. 


0 1,1  - <7 1.o  ” 1 - <7 1.1  “ exp(-0,/2). 

exp(-a,/2) 


1 - exp(-0,/2)  + exp(-0|)  ’ 


The  headway  distribution  (2.10)  becomes 
F(h)  - 


0 < h < a 2, 

(1  - Po)exp(-/i/2)-p0exp(-/i),  a2<  h < a t 
p0exp(-/i),  a 2 < h 


from  which  one  can  evaluate  £(//),  E(H 2),  and  E(W)  as  explicit  functions  of  0|  and  a2- 
The  pair  of  equations 


8f(W0 
do  | 


dE(W) 


da 


- 0 for  a | - a [ and  a 2 - a 2\ 


for  the  optimal  at  and  a2<  can  be  manipulated  into  the  form 
(3.2a,b)  a'\  " “ 21n(exp(— a j/2)  - 022/8],  a[  - E*(W). 


The  minimum  of  E(W)  can  again  be  obtained  by  successive  approximations.  If  we  sub- 
stitute any  yth  approximation  a^j)  in  (3.2a),  we  obtain  a /h  approximation  a\(J).  The 
0'+l)th  approximation  for  a2  is  then  obtained  by  substituting  a\(J)  and  a\(J)  into  E(W ),  as 
in  (3.2b).  This  sequence  of  approximations  converges  very  rapidly.  Table  1 illustrates  the 
iterations  starting  from  02‘(O)  - 1/2. 
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TABLE  1 


Iteration  Number 

0 

1 

2 

a 2' 

0.500 

0.899 

0.838 

a'\ 

0.582 

1.243 

1.125 

Po 

0.922 

0.715 

0.755 

£*(//) 

1.091 

1.366 

1.310 

£*(  (F) 

0.899 

0.838 

0.835 

Whereas  the  optimal  control  for  A - 1 reduced  £((F)  from  1 to  0.901,  £((F)  can  be 
further  reduced  to  0.835  for  A - 2.  Again,  as  in  the  case  of  one  vehicle,  this  result  is  obtained 
by  an  increase  of  E(H)  from  1 to  1.31  (approximately  the  same  as  for  A — 1)  and  by  a larger 
decrease  in  C2(//),  from  1 to  0.28. 

For  A > 2,  it  becomes  progressively  more  tedious  to  determine  the  a/s  by  setting 
derivatives  of  £(  (F)  equal  to  zero.  One  should,  instead,  use  numerical  techniques  better  suited 
for  a computer.  For  any  A and  any  particular  choice  of  the  a/s,  the  matrix  qxv  was  computed 
numerically  from  (2.7)  and  (2.8).  The  system  of  A linear  equations  (2.4)  was  then  solved  for 
the  px.  That  qxv  — 0 for  y < x — 1 means  that  the  yth  equation  of  (2.4)  can  be  solved  for 
pv+ 1 in  terms  of  p0,  . . . , p„  y < A — 1.  Thus,  for  y — 0,  1,  . . . , one  can  successively  evalu- 
ate p |,  Pi.  . . . , in  terms  of  p0 , then  determine  p0  from  the  normalization.  The  headway  distri- 
bution can  be  evaluated  from  (2.10),  and  the  values  of  £(//),  £(//2),  and  £(  (F)  calculated. 

The  evaluation  of  £((F)  for  any  particular  A and  (a*)  is  computationally  quite  simple 
even  for  moderately  large  A.  To  minimize  £((F)  with  respect  to  the  {ax},  however,  we 
exploited  certain  known  properties  of  the  a’,  such  as  (2.14),  and  followed  an  iterative  scheme 
for  which  £(  (F)  would  converge  rapidly  to  £*(  (F).  The  scheme  successively  minimizes  £(  (F) 
with  respect  to  one  variable  at  a time  in  the  following  way. 

First  we  take  a2(0)  “ <*3(0)  - ...  - aN( 0)  - 0.5  and  let  a , vary  over  the  interval  [0.5, 
2.5)  (in  steps  of  0.1).  We  find  the  best  £(  IF),  £(  IF)  (0),  and  the  corresponding  aj(0).  Next 
we  choose  a,(l)  - a,(0)  and  a3(l)  - a4(l)  - ...  - a*(l)  - £( (F)(0)  and  let  a2  vary  over 
the  interval  [a3(l),  a,(l)).  We  find  the  best  £((F),  £(  (F)(1),  and  the  corresponding  a2(l). 
In  the  third  step,  we  choose  a, (2)  — a ,(1)  a2(2)  — a2(l),  and  a«(2)  — . ..  - aw(2)  — 
£(  (F)(1)  and  minimize  £(  (F)  with  respect  to  a 3. 

We  continue  until  we  have  minimized  £((F)  with  respect  to  aN. lt  chosen 
aN  - £((F)  (A  - 2),  and  obtained  a new  trial  sequence  aN(N  - 1)  < (A  - 2)  ^ 

...  < aj(0).  Starting  with  this  trial  solution,  we  now  repeat  the  above  procedure  of  succes- 
sively determining  new  values  of  a,,  a2 av.  As  was  true  in  the  numerical  schemes  for 

A - 1 and  2,  this  procedure  converges  within  only  a few  cycles. 

It  took  only  1.6  minutes  for  a CDC  6400  computer  to  run  the  program  for  A - 15.  This 
time  would  likely  increase  rapidly  with  A.  Our  purpose  here,  however,  was  mostly  to  bridge  the 
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gap  between  the  results  for  N = 1 and  2 and  the  asymptotic  results  described  in  the  next  sec- 
tions. No  attempt  was  made  to  develop  more  efficient  programs  for  dealing  with  larger  N 
because  the  practical  limitations  of  the  model  do  not  justify  high  precision. 

Results  of  these  calculations  are  shown  in  Figures  2,  3,  4,  and  5.  Figure  2 shows  values 
of  £*(  WO  as  a function  of  N.  Although  these  are  defined  only  for  integer  N,  the  points  at 
integer  N have  been  joined  by  a smooth  curve.  The  unit  of  time  (2.1)  has  been  chosen  as  the 
average  uncontrolled  headway,  not  the  trip  time.  If  one  were  simply  to  add  more  vehicles  to  an 
existing  route,  keeping  E(T)  constant,  then  the  average  wait  in  real  time  units  would  be 
£*(  W)E(T)/N  which  will,  of  course,  decrease  with  N at  a much  faster  rate  that  £*(  WO.  The 
reason  why  £*(  WO  decreases  with  N is  that,  the  larger  N is,  the  less  the  control  of  any  one 
headway  affects  the  future  arrivals.  All  uncontrolled  systems  give  a Poisson  arrival  process  of 
vehicles,  but  the  larger  N is,  the  larger  is  the  space  of  control  strategies.  For  N — > °°,  the 
optimal  control  will  permit  nearly  regular  headways,  and  £#(  WO  = 1/2,  as  compared  with  the 
uncontrolled  strategy  with  £^(  WO  = 1. 


Figure  3 illustrates  the  distribution  of  controlled  headways  for  N - 1,  2,  3,  IS,  and  °°, 
and  also  the  common  exponential  distribution  for  the  case  of  no  control.  We  expect  that,  for 
N — oo,  the  optimal  control  will  produce  H - 1 with  probability  1 because,  by  keeping  only  a 
finite  number  of  vehicles  at  the  depot,  one  can  make  the  probability  of  running  out  of  vehicles 
(Po)  arbitrarily  small.  One  can  see,  however,  that  the^ approach  of  F*(h)  toward  that  of  N - °° 
is  quite  slow.  Figure  3 also  shows  an  approximate  £(/>)  for  N - 1000  obtained  from  asymp- 
totic formulas  derived  in  the  next  sections. 

Figure  4 illustrates  the  optimal  strategies  a'  for  several  values  of  N.  Since  vehicles  are 
dispatch  at  a rate  !/£(//),  this  is  also  the  rate  of  return.  For  small  x,  x - 1.  2.  ....  one 


....-v  ; -.v  i 
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dispatches  with  headways  larger  than  £(//)  to  prevent  the  state  from  reaching  0,  but  if  one  has 
many  vehicles  in  the  depot,  one  dispatches  them  more  frequently. 

Figure  5 shows  the  distribution  of  the  number  of  vehicles  available  in  the  depot  immedi- 
ately after  a dispatch  for  the  cases  of  (V  — 1,  2,  3,  and  15.  The  distribution  is  drawn  with  coor- 
dinate y - iV_l/3x  in  anticipation  of  the  limit  behavior  for  A'  — °°  described  in  the  next  sec- 
tions. 


Figure  5 — Distribution  of  the  number  of  vehicles  in  the 
depot  immediately  after  a dispatch. 


4.  A CONTINUOUS  APPROXIMATION 

Figures  3,  4,  and  5 suggest  that  for  N » \,  the  a,*'sand  p "'s  change  relatively  little  as  x 
changes  by  1.  It  seems  advantageous,  therefore,  to  change  the  domain  of  x from  the  integers 

(0,  1 N)  to  the  continuous  interval  [0.  N\  and  to  assume  that  ax  of  Figures  1 or  4 can  be 

approximated  by  a smooth  curve  a(x)  and  that  px(k)  can  be  approximated  by  a smooth  func- 
tion p(x,  k).  The  p(x,  k),  which  can  be  interpreted  as  a "probability  density"  of  the  number 
of  vehicles  at  the  depot  immediately  after  the  kth  dispatch,  will  satisfy  a continuous  counterpart 
or  (2.3) 

(4.1)  Piy.  k+ 1)  - fQ  q(x,y)  p(x.k)  dx, 

in  which  q(x,y)  is  the  conditional  probability  density  of  having  y vehicles  after  the  (fc+l)th 
dispatch,  given  that  there  were  x vehicles  after  the  kth  dispatch. 

We  anticipate  that,  for  N » 1,  the  effective  "width"  of  the  distribution  p(x,k)  will  be 
large  compared  with  the  probable  one  step  changes  of  the  state,  y - x,  and  that  the  p(x,  k) 
will,  therefore,  approximately  satisfy  a diffusion  equation  of  the  form  [2]* 

(4.2)  p(x,  *+l)  - p(x,  k)  - - [«(x)p  (x.rfr)]  + [0(x)p(x,  *)]. 

dx  2 dx 2 

*In  reference  [21,  ihe  left-hand  side  of  (4.2)  would  be  written  as  dp{x,k) /dk. 
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in  which 

(4.3)  a(x)  = E(Y  — x|x)  and  /3(x)  = £((  K — x)2|x) 

are  the  first  and  second  moments  about  xof  the  random  variable  Y\x  having  a probability  den- 
sity f>(x,  y). 

For  k — °°,  we  expect  p(x,  k)  to  approach  an  equilibrium  distribution  p(x)  which  is  a 
solution  of  the  ordinary  differential  equation 

(4.4)  - -f  Mx)  p(x)]  + | [/3(x)  p (x ) ] = 0. 

dx  2 dx2 

We  also  expect  for  N » 1 that,  under  an  optimal  strategy,  there  should  be  a negligible  proba- 
bility for  the  number  of  vehicles  at  the  depot  to  be  comparable  with  N , i.e.,  p(x)  and  any 
derivatives  vanish  for  sufficiently  large  x.  Thus  we  can  integrate  (4.4)  to  obtain 

(4-5)  - [a(x ) /j(x)]  + y d[) 3(x)  p(x)]/dx  *=  0, 

which  in  turn,  gives 

(4.5a)  /3(x)  p(x)  = exp  2 J [a(y)//3(y)l  rfyj 

for  some  integration  constant  x^,  which  is  to  be  determined  by  the  normalization  condition 
(4.5b)  p(x)  dx  ~ p(x)  dx  = 1. 

The  a(x)  and  /3(x)  can  be  expressed  in  terms  of  the  control  strategy  a(x);  therefore 
(4.5a)  represents  .ssentially,  the  continuum  approximation  to  the  solution  of  the  N simultane- 
ous equations  (»  w.  The  a(x),  however,  must  eventually  be  chosen  so  as  to  minimize  E(W), 
which  can  also  be  expressed  in  terms  of  the  p(x)  and  a(x). 

Actually,  instead  of  using  (4.5a)  to  express  p(x)  in  terms  of  a(x),  so  as  to  minimize 
FAW)  with  respect  to  a(x),  we  will  use  (4.5)  to  express  a(x)  in  terms  of  p(x)  and  minimize 
£(  W)  with  respect  to  p(x).  Whereas  in  the  discrete  case  we  minimized  £(  W)  with  respect  to 

the  N parameters  ax,  a7 fl/v,  the  continuum  version  will  involve  minimizing  E(W)  with 

respect  to  a function  a(x)  or  p(x),  and  give  a calculus  of  variation  problem. 

5.  CALCULATION  OF  a(x)  AND  0(x) 

The  exact  transition  probabilities  qK  V are  given  in  terms  of  av  by  (2.7)  and  (2.8).  Since  ax 
is  to  be  approximated  by  a smooth  function,  we  expect  that  ax  — ax+\  will  be  small  compared 
with  a,.  It  is  convenient  therefore  to  note  that  transition  from  xto  x + / can  occur  from  any  of 
the  mutually  exclusive  events 

A = exactly  /+1  arrivals  in  [0,  at+, J 

B = exactly  /+1  arrivals  in  [0.  a(+,+l)  and  1 or  more  arrivals  in  (a,+/+1.  at+/],  or 

C = at  most  / arrivals  in  [0,  ar+,+I)  and  at  least  1+2  arrivals  in  [0,  av+,]. 

Event  B involves  at  least  one  arrival  in  (at+/+i  ax+j\,  and  C involves  at  least  two.  For 
small  values  of  ax+/  -0t+/+l,  we  can  write 


Jsri 
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p(A)  “ (^1  [1  - exp(-af+,//V)]'  + 1 lexp(~ax+l/N)) 

P(B)  " (^T*)  11  - exp(-0r+/+1//V)'+l  [exp(-flx+/+l/yv)j 

x (ax+i  ~ ox  s,+|)  + 0(ax+i  ~ a,+/+l)2 
= ~ a»+/+i)  p(A)  + 0 (ax+/  — ar+/+|)2, 


p(C)  » O(0X+/  — ax+/+i)2, 

therefore 

(5.1)  <7*.*+/  “ (T+l*)  ^ “ exP(_ax+//^)],+l[exp(-at+//iV))A'_)f~,~l 

((  “1"  (^x+/  ^v+/+|)J  + 0(ax+/  aV4./+j)2,  / ^ — 1. 


The  conditional  moments  a(x)  and  /3(x ) can  be  evaluated  from 

(5.2)  a(x)  = £ /<7,.x+/  and  0(x)  = £ /2<7XJt+„ 

/—i  /— i 

but  these  are  difficult  to  evaluate  exactly,  because  ax+l  is  a function  of  /.  To  simplify  them,  we 
expand  ax+,  - a(x+/)  as 

ax+l  « a (x+1)  - a (x+1)  + (/-l)a'  (x+/)  + — a"  (x+1)  + ...  , 

where 


a'(x+l)  = a (x+1)  and  a"(x+l)  - -f-  a(x+l). 
ax  dx  2 

If  we  expand  the  terms  of  a(x)  and  /3(x)  in  powers  of  a'(x+l),  a"(x+l),  x.  x/N,  aJN , 
etc.,  the  sum  over  / can  be  evaluated  term  by  term  from  the  moments  of  a Poisson  distribution. 
For  purposes  of  combining  terms  of  the  same  order  of  magnitude,  however,  it  is  convenient 
here  to  anticipate  the  relative  magnitudes  of  x.  a'(x+/),  etc.  We  assume  now,  and  verify  later, 
that  the  relevant  range  of  x is  x — 0(W  ’ /3)  for  N » 1,  and  that  in  this  range  of  x (except  pos- 
sibly for  x — o(Wl/3)), 

(5.3)  a (x+1)  - 1 + 0(A/-,/3).  a'(x+l)  - 0(Ar2/3).  a"(x  + 1)  - 0 (/V1). 

That  a (x+1)  should  be  approximately  one  derives  from  the  fact  that  one  must  have  an  average 
of  one  arrival  between  each  dispatch  to  maintain  an  equilibrium.  An  a(x)  > I means  that 
vehicles  are,  on  the  average,  returning  faster  than  they  are  being  dispatched. 

An  expansion  of  (5.2)  in  powers  of  N gives,  for  N » 1, 

a(x)  - a(x  + 1)  - 1 - (x/N)  + 0(AM) 

0(x)  - a (x+1)  + 0(/V'2/})  - 1 + 0(ATl/3). 


(5.4) 
and 

(5.5) 
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6.  THE  EXPECTATION  OF  WAIT 

To  express  E(H),  £(//*),  and  E(W)  in  terms  of  a(x),  p(x),  we  write  £(//)  in  the 

form 

, v /V  7+1 

<61)  £(//)-  £ ±E(H\ i.j)qup, 

/ - 0 /-0 

where  E{H\i,j)  is  the  expectation  of  a headway  that  starts  with  / vehicles,  and  ends  with  j vehi- 
cles after  the  dispatch,  and  q,  ,p,  is  the  joint  probability  of  having  i and  j such  vehicles. 

Since  the  headway  that  ends  with  j vehicles  after  the  dispatch  is  usually  a/+),  and  al+]  is 
close  to  one  for  most  j,  we  rewrite  (6.1)  in  the  form 


N /+l 

£(//)  - £ £ [1  + (a/+]  - 1)  + E(H  - a/+1|/,7)l  q,,t  p, 

i-0  i-0 

We  know  that 

7 + 1 

L <?/./  Pi  - P/- 
/-0 

and  £(//  - a,+l\j+l,j)  - 0, 

therefore 

(6.2) 

£(//)  - 1 + £ (a;+1 

7-0 

- l)p;  + £ £ E(H  - aJ+i\i.j)qup,. 
i-0  ,-o 

Similarly, 

(6.3) 

£(//2)  - 1 + £ (ay2+| 
/-0 

- Dp,  + £ £ E(H2  - a(+|  |/',7)  <7,/  Pr 
I-o  1-0 

We  anticipate  that  the  first  term  of  (6.2)  will  be  large  compared  with  subsequent  terms.  If 
we  expand  1 /£(//)  about  one,  and  multiply  it  by  £(//J),  we  obtain 

N N 

(6.4)  2£(  W)  - 1 + £ (a4,  - Dp,  - £ (a/+1  - Dp, 

i-o  i- o 

+ £ t IE(H2  - aj\t  | ij)  - E(H  - am\ij)]q,.>  p,  + R 

J-0  i-0 
N 

~ 1 + £ (fl/+ 1 — l)fl/+i  Pi 
i-o 

+ £ £ E[(H  - a,+i)  ( H + a/+i  - l|/J]  q,,  P,  + R 

7-0  »—0 

in  which  R contains  terms  proportional  to  second  or  higher  powers  of  [E(H)  - 1]  and 
[£(//*)  - 1]. 

In  order  for  H to  deviate  from  a/+1,  it  is  necessary  that  there  be  at  least  one  arrival  in 
[aj+i,  a,].  Therefore, 

(6.5)  £l(//— a/+1)  (W+a/+l  - 1)M 

- £[(//-«,+,)  (//+a/+i  - 1)| IJ  and  //*a/+1]  P (H*at+x\iJ) 
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is  of  order  (a,  — a/+ |)2.  Except  possibly  for  the  first  few  values  of  j,  say  j < y — o(Nl/}),  we 
expect  from  (5.3)  that  (a,  — al+l )2  ■ — [a'O')]2  — These  terms  will  give  a negligible 

contribution  to  (6.4). 

The  terms  in  (6.5)  for  small  i.j  are  not  necessarily  negligible,  particularly  the  term  for 
i - j - 0,  because  a„  - °o.  If  we  started  with  no  vehicles  in  the  depot  and  no  vehicle  returned 
by  a |,  we  would  have  to  wait  until  the  arrival  of  the  first  vehicle.  If  this  extra  waiting  occurs, 
we  would  wait  on  the  average  another  average  arrival  headway.  This  term  of  (6.5)  represents 
the  penalty  for  running  out  of  vehicles,  which  a proper  control  strategy  will  want  to  avoid. 

We  could  consider  separately  the  terms  of  (6.4)  that  contain  the  p,  with  j > y from  those 
with  j > y , and  write  (6.4)  in  the  form 

(6-6)  2E(  W)  « 1+  £ (ai+i  - 1)  al+lp,  + "l  b,  p,  + R + 0((V4/3). 

l-y  i-O 

in  which  the  6,  are  positive  functions  of  the  a/s. 

For  j > y,  we  can  approximate  the  a/+)  and  p , by  the  continuous  functions  a(x+l)  and 
p(x)  and  write  (6.6)  as 

(6.7)  2 E(  W)  ~ 1 + J*  la(x  + 1)  - Ua(x+1)  p(x)  dx  + T b.  p,  + R. 


The  second  term  of  (6.7)  involves  both  the  a(x+l)  and  p(x),  but  these  two  functions 
are  related  through  (4.5),  (5.4),  and  (5.5).  We  can  use  these  equations  to  express  a(x+l)  in 
terms  of  p(x).  From  (5.4)  and  (4.5),  we  have 


a (x+1)  - 1 ~ a(x)  + — 
N 


Thus  the  integrand  of  (6.7)  becomes 

(6.8)  p(x)  a (x+1)  [a(x+l)  - 1]  = p(x)  (a(x+l)  - 1]  + p(x)  (a(x+l)  - l)2 


-r  -j-  (/3(x)p(x) I + p(x)  + p (x ) 
l dx  N 


_1_  ± (,(x)  ,(x)]  + i 


The  first  term  of  this  is  of  the  largest  order  relative  to  N , but  it  is  a perfect  differential  and 
integrates  to 


4-1/2 


__  l , l, 

j P(y-  j) 


1 

, i 

py- t 

1 + a y + y 

- 1 

i j.  1 d , 1 , 

2 p(y  — 1/2)  dy  P y 2 


= -y  P(y). 


In  the  last  term  of  (6.8),  we  can  approximate  /3(x)  by  one  and  neglect  the  x/N,  so  that 
(6.7)  becomes 


(6.9)  2 E(W)  ^ 


— p(x)  + 


1 

4p(x) 


dp(x) 

dx 


dx  - j P(y)  + L *//>/  + R 

1 i- o 


u. 
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Although  we  have  used  the  equations  relating  the  a,'s  to  the  p,'s  to  eliminate  a(x  + \) 
from  the  integral  of  (6.7),  the  ph  bh  and  a,  for  j < y are  still  constrained  to  satisfy  (2.4).  The 
details  of  this  are  quite  complex,  but  the  issues  are  clear. 

The  p,  must  satisfy  the  normalization  condition 

(6.10)  2>/  + Sy-U2pix)  dx  ~ 

i-o  y ' 

and,  through  (2.4),  the  p(y)  in  (6.9)  depends  upon  the  /?,,  0 ^ y-1.  By  appropriate  con- 

trol, we  can  distribute  the  probabilities  p , so  that  either  most  of  the  probability  lies  in 
0 < j < y - 1,  or  most  lies  in  j > y.  The  term  —(1/2 )p(y)  in  (6.9)  suggests  that,  to  minim- 
ize (6.9),  one  should  assign  most  of  the  probability  to  small  j so  as  to  make  this  term  as  large 
as  possible.  One  can  show  that  this  term  is  identified  with  the  fact  that  the  smaller  j is,  the 
more  vehicles  there  are  in  use,  and  consequently  the  shorter  the  average  headway  is. 

On  the  other  hand,  the  terms  b/Pj  of  (6.9)  for  0 < j < y-1  are  all  positive,  particularly 
the  term  for  y - 0,  which  includes  an  extra  wait  if  j “ 0 and  no  vehicle  has  returned  by  the 
time  a,,  when  a dispatch  is  desired.  To  minimize  these  terms,  one  should  have  a very  small 
probability  for  j < y — 1.  One  can  show  that  the  effect  of  these  terms  overpowers  the  term 
-0/2)/?  (y),  and  dictates  a policy  for  which  the  p ; for  y < y - 1 are  small  for  N » 1. 

For  sufficiently  larger  N,  the  strategy  which  minimizes  £(  WO  must  be  approximately  one 
for  which  p(x)  is  chosen  so  as  to  minimize  the  second  term  of  (6.9)  with  y ~ 0,  i.e., 
minimize 


(6.11) 

subject  to 
(6.1  la,b) 


'“f 

Jo 


dp(x) 


dx 


dx, 


p(0)  - 0 and  f p(x)  dx  - 1. 


7.  LIMIT  DISTRIBUTION 

To  minimize  (6.11)  subject  to  (6.1  la, b)  is  a standard  calculus  of  variation  problem.  It  can 
be  transformed  into  a more  familiar  form,  however,  if  we  let 


(7.1) 


p(x) 


N 


— f2 


1/3 


Nl/1 


. y 


N\n 


Then,  in  terms  of  /(y),  minimize  (6.11)  subject  to  (6.11a,b)  becomes  the  well  known  Sturm- 

Liouville  Problem  [3]: 

minimize 

+ |^‘J’ 


(7.2) 


j _ _J — r 
J /v2/3  Jo 


y fHy) 


\ dy 


dy 


subject  to 
(7.2a, b) 


/(0)  - 0 and  /J(y)  dy  - 1. 
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l 

! 


The  Euler  equation  corresponding  to  minimize  (7.2)  subject  to  (7.2b)  is 


(7.3) 


d2r(y) 

dy 2 


(y  4-  A)  /*(.y). 


which  is  the  Sturm-Liouville  differential  equation  with  Lagrange  multiplier  A.  The  only  solu- 
tion of  (7.3)  that  vanishes  for  y — <»  is 


/•O')  = K Ai  O + A) 


in  which  K is  a constant  and  Ai()  is  the  Airy  Function  [1],  The  normalization  (7.2b)  deter- 
mines A;  thus 

(7  4)  f*(y)  = AiO  + A)/{ dt;V12 

and 


(7.5) 


P*(x) 


N 


1/3 


2 

Ai 


2 

Ai(f)  dt 


This  determines  f*(y),  except  for  a translation  of  coordinates  by  A,  but  the  boundary 
condition  (7.2a)  requires  that  Ai(A)  = 0.  Figure  6 shows  graphs  of  Ai(z),  Ai(z)  and 

Ai '(z)/A(z)  = d In  Ai  {z)/dz. 


Although  Ai(z)  has  zeros  at  z 2.339,  —4.088 we  know  from  (4.5)  and  (5.4) 

that 

a(N'ny  + 1)  = 1 + _ \ + ± |n  Ai(y  + A), 

2 p(x)  dx  dy 
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and  that  a(x)  must  be  a monotone  decreasing  function  of  x As  one  can  see  from  Figure  6, 
values  of  v + A less  than  -2.339  would  violate  the  latter  condition,  consequently  A must  be  the 
zero  at 

(7.6)  A = —2.339. 

The  normalization  in  (7.4)  and  (7.5)  can  be  evaluated  by  numerical  integration: 


J*  Ai<*)  = 0.491 


Thus  the  optimal  limit  distribution  and  control  are 
(7.7)  „»i„\  ~~  inn  m -Mi  a 


P*(x)  ~ 2.037  N 1/3  Ai(x\-'/}  —2.339) 


n*(x  + l)  = 1 + \-,n  Ai'(x/V~l/3  -2.339)/Ai(xN-|/3  -2.339). 


This  verifies  the  conjecture  made  in  (5.3),  particularly  that  a(x+l)  - 1 + 0(A(_1/3). 
Indeed  the  dependence  of  />*(x)  and  a*(x+l)  - 1 upon  N involves  merely  a scaling  of  coordi- 
nates with  all  lengths  x measured  relative  to  Nl,}.  For  large  values  of  xN~u,%  one  can  use  the 
asymptotic  approximations 

(78a)  Ai(z)-— ^z'^exp  ~|*3/2  \ z-J"+ ... 

i 27 r 3 48 


(7.8b)  Ai  (z)/Ai(z)  — — z+1/2+  (—1/4 )z  ' + for  z » 1, 

and,  for  small  values, 

(7.9a)  Ai (z ) = (0.491)  (z  + 2.339) 

(7.9b)  Ai  (z)/Ai(z)  = (z  + 2.339)  for  (z  + 2.339)  « 1. 

Thus,  p*(x)  decays  very  rapidly  for  large  values  of  xN~x/i,  and  is  proportional  to  x2  for  small  x. 

The  mode  of  the  distribution  p*(x)  occurs  where 

Ai  (xAM/3  + 2.399)  - 0.  x yv1/3  = 1.320. 

At  the  mode,  the  second  term  of  (7.8)  vanishes,  i.e.,  a*(x  + 1)  = 1,  which  means  that  the 
system  is  "in  balance,"  with  a dispatch  rate  nearly  equal  to  the  arrival  rate.  The  expected 
number  of  vehicles  in  the  depot  immediately  after  a dispatch  can  be  evaluated  by  numerical 
integration  to  give 

£U)  = (1.559)  NUi. 

somewhat  larger  than  the  mode  because  of  the  skewed  distribution. 

Of  the  N vehicles  one  has  available,  the  fraction  of  vehicles  which  one  keeps  at  the  depot 
is  thus  of  order  /V-2/3. 

The  smooth  curve  of  Figure  5 shows  the  complementary  distribution  function 


f p*(x')  dx'  - / Ai(y  - 2.339)  dy' 
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as  a function  of  xN~i,}.  One  can  see  that  the  discrete  distributions  for  N - 1,  3,  and  15  are 
approaching  this  limit  distribution  but,  since  these  asymptotic  expansions  are  essentially  expan- 
sions in  powers  of  jV_1/3,  one  does  not  expect  very  rapid  convergence  relative  to  N.  The  main 
error  seems  to  be  associated  with  the  fact  that,  for  finite  N,  one  can  allow  a nonzero  probability 
for  x — 0 with  a moderately  large  av  For  N — 15,  p0  is  still  about  0.14.  To  obtain  a second 
approximation  in  the  asymptotic  expansions  is  very  tedious;  attempts  to  do  so  were  not  very 
fruitful.  One  can  see  from  Figure  5 that  a translation  x — » x + 1 would  improve  the  fit  consid- 
erably, but  the  asymptotic  expansions  cannot  be  expected  to  distinguish  between  x - 0 and 
x - 1. 

Figure  7 compares  a** for  \ - 15  with  the  continuous  a*(x).  On  this  scale,  0 < x < 15, 
the  agreement  does  not  look  very  good,  but  most  of  the  state  probability  lies  in  the  range  x - 1 
to  5,  where  the  agreement  is  satisfactory. 

X Numbar  of  Vahiclaa 


Since  the  headway  which  ends  with  x vehicles  in  the  depot  is  approximately  a*(x  + 1), 
and  the  p*(x)  are  known,  one  can  easily  evaluate  a continuum  approximaton  to  the  headway 
distribution  F*(h).  The  curve  of  Figure  3 for  N - 1000  is  obtained  in  this  way.  This  clearly 
demonstrates  how  slowly  this  distribution  converges  to  the  limit  H = 1 . 
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By  substituting  (7.4)  into  (7.2),  one  can  evaluate  the  integral  7A'2/1,  and  show  that 
2 £•(  W)  ~ 1 + (2.282)  AT2'3  + oOV'2'3). 

In  Figure  2,  this  continuum  approximation  is  represented  by  the  curve  labeled  E'ci  IF). 

8.  DISCUSSION 

The  main  purpose  of  the  above  analysis  was  to  investigate  the  nature  of  efficient  control 
strategies  for  a public-transportation  system  having  a large  number  N of  vehicles  serving  the 
same  depot.  The  measure  of  performance  is  considered  to  be  the  average  wait  of  passengers  at 
the  depot. 

Previous  work  [4]  has  described  strategies  (at  least  for  small  AO  in  which  the  trip  time  of 
vehicles  is  so  predictable  that  vehicles  do  not  pass  enroute.  The  analysis  of  optimal  control 
strategies  for  several  vehicles  with  some  passing  is  extremely  difficult.  The  optimal  strategy 
would  be  very  complex  since  it  would  exploit  any  relevant  information  about  past  departure 
times  of  all  vehicles.  It  seem  reasonable,  therefore,  to  consider  cases  with  large  N as  some 
indication  of  how  the  qualitative  nature  of  the  strategy  changes  when  the  fluctuations  in  trip 
time  become  so  large  that  enroute  passing  of  vehicles  becomes  a dominant  feature. 

The  postulate  of  exponentially  distributed  trip  time,  which  was  made  to  simplify  the 
mathematics,  is  obviously  not  representative  of  typical  transportation  systems,  which  usually 
have  a coefficient  of  variation  in  trip  time  considerably  less  than  one.  The  analysis  of  sections  2 
and  3 is  therefore  of  questionable  relevance  to  real  applications  except  as  a crude  upper  bound 
on  the  effect  of  fluctuations  or  possibly  as  a first  step  in  the  mathematical  analysis  of  systems 
with  more  general  (possibly  Erlang)  distribution  of  trip  time. 

The  results  of  Sections  4 to  7,  particularly  the  conclusion  that  the  optimal  number  of 
vehicles  to  keep  at  the  depot  is  0(A(I/3),  though  rather  crude  even  for  an  exponential  trip-time 
distribution,  are  likely  to  be  quite  insensitive  to  the  trip-time  distribution  for  N » 1.  They 
should  provide  an  cpper  bound  on  the  effects  of  fluctuations,  and  possibly  a close  one.  The 
reason  that  one  should  expect  this  is  as  follows. 

Even  though  vehicles  are  dispatched  with  nearly  regular  headways  under  the  optimal  con- 
trol, if  N » 1 and  NCiT)  » 1,  vehicles  will  pass  each  other  enroute.  The  process  of 
returning  vehicles  is  the  superposition  of  the  processes  generated  by  each  of  the  N vehicles 
and,  under  quite  general  conditions,  will  have  stochastic  properties  similar  to  a Poisson  process. 
In  particular,  in  a time  r « NCiT)  the  number  of  returning  vehicles  should  have  approxi- 
mately a Poisson  distribution,  because  it  is  a count  of  statistically  independent  rare  events  (the 
return  of  any  Ah  vehicle).  During  such  time  intervals,  the  return  process  is,  therefore, 
insensitive  to  the  stochastic  properties  of  T , except  for  the  mean  N. 

From  (4.2)  one  can  show  that  the  "natural  unit  of  time"  for  the  diffusion  equation  is  of 
order  N2n.  This  is  the  time  it  takes  for  the  system  to  reach  an  equilibrium  or  to  return  to  an 
equilibrium  after  some  disturbance.  If  the  return  process  behaves  like  Poisson  process  for 
times  of  this  order,  i.e.,  if  r = Nin  « NCiT)  or  CiT)  » N~l/3,  one  might  expect  the 
diffusion  equation  to  be  approximately  correct.  Of  course,  N~in  is  not  very  small  for  reason- 
able values  of  N. 

The  results  described  here  may  be  useful  as  a bound  on  the  strategies  for  real  systems, 
but  they  are  not  expected  to  be  quantitatively  accurate  for  any  of  the  systems  which  motivated 
this  analysis.  The  intended  applications  were  for  bus  routes  (downtown  terminal  to  suburbs,  or 
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airport  to  downtown)  with  N perhaps  about  20  and  C(T)  probably  about  0.1,  or  elevators  with 
N about  6 and  a C(T)  perhaps  about  0.3.  Unfortunately,  these  values  of  Ware  not  large 
enough  for  the  present  theory  to  be  accurate. 
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University  of  Southern  California 
Los  Angeles,  California 

ABSTRACT 

A definition  of  the  problem  of  the  initial  transient  with  respect  to  the 
steady-state  mean  value  has  been  formulated  A set  of  criteria  has  been  set 
forth  by  which  the  efficacy  of  any  proposed  rule  may  be  assessed.  Within  this 
framework,  five  heuristic  rules  for  predicting  the  approximate  end  of  transien- 
cy, four  of  which  have  been  quoted  extensively  in  the  simulation  literature, 
have  been  evaluated  in  the  M/M/1  situation.  All  performed  poorly  and  are 
not  suitable  for  their  intended  use 


1.  INTRODUCTION 

In  this  paper  we  consider  some  of  the  problems  posed  by  the  existence  of  an  initial  tran- 
sient in  the  system  response  which  may  arise  in  a digital-computer  simulation  of  a stochastic 
process.  Basically,  the  situation  is  this.  Suppose  that  a discrete-parameter  stochastic  process 
{X„  l - 1,2,3,  ...  } is  being  observed  for  which  a set  of  initial  conditions,  denoted  by  /,  exist 
at  i ” 0.  For  example,  X,  may  be  inherently  discrete,  like  the  waiting  time  of  the  r,h  customer 
arriving  at  a queuing  system  after  the  simulation  begins;  or,  it  may  arise  by  sampling,  at 
equidistant  time  intervals,  a continuous  time  series  such  as  the  number  of  jobs  in  a system. 
We  suppose  that  the  first  moments  of  these  random  variables  exist  and  tend  to  an  asymptotic 
limit,  independent  of  /,  i.e., 

lim  E[X,\I\  - Moo. 


where  m-  is  defined  as  the  steady-state  mean.  We  assert  that  the  principal  problem  of  the  ini- 
tial transient  is  that  of  determining  the  minimum  i,  call  it  /*,  such  that  the  expectation  of  the 
random  variables  X„  t > t\  is  as  close  as  one  desires  to  the  limiting  expectation. t Symbolically, 
i*  is  the  smallest  / for  which 


1 - « < 


E[X  J 


< 1 + «. 


i 


‘This  research  was  supported  in  part  by  the  National  Science  Foundation  under  Grant  Number  ENG  75-06900 
tThroughout  this  paper  we  use  the  standard  notation  of  an  upper-case  letter  for  a random  variable  and  a lower-case 
letter  for  a specific  value  the  random  variable  may  assume. 
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where  FAX A = /x«>  is  the  steady-slate  expected  value  and  e>0  is  a preassigned  number.  Thus, 
for  example,  t = 0.05  if  one  desires  to  be  within  five  percent  of  the  steady-state  mean  value. 
Figure  1 illustrates  examples  of  some  ways  that  E[X]/ E[X  A may  converge  and  also  the  i* 
corresponding  to  the  given  e.  Though  our  notation  does  not  show  it  explicitly,  it  should  be 
noted  that  r* depends  on  e. 


Figure  1 — Some  ways  fl^l/Fl-V^l  converges  and  (he 
associated  /•  value  for  the  indicated  e value 


The  importance  of  this  problem  arises  from  the  fact  that  a simulator  must  know  the  con- 
ditions under  which  data  are  being  collected.  Is  the  process  near  the  limiting  condition  of 
steady  state,  or  does  it  still  have  a long  way  to  go  to  be  near  the  equilibrium  condition?  That 
question  must  be  answered  in  order  to  collect  and  analyze  data  appropriate  for  any  specific  pur- 
pose. For  example,  if  all  data  are  collected  far  from  equilibrium  they  cannot  be  used  to  pro- 
duce good  estimates  of  the  steady-state  mean.  The  disastrous  consequences  of  not  properly 
accounting  for  the  initial  transient  have  been  illustrated  very  well  by  Law  [5)  in  the  construc- 
tion of  a confidence  interval  for  the  steady-state  mean,  using  either  the  method  of  replication  or 
the  method  of  batch  means.  Knowledge  of  z*  is  not  only  necessary  for  estimation  of  steay-state 
parameters,  but  also  for  parameters  of  the  transient  part  of  the  process.  In  the  latter  case, 
repetitive  samples  are  required,  and  it  is  wasteful  to  go  past  r*or  inadequate  to  be  far  short  of 
it. 

Several  methods  or  rules  of  thumb  for  determining  an  estimate  of  /*  have  been  sug- 
gested in  the  literature  and  are  discussed  later.  It  was  anticipated  that  these  methods  for  detect- 
ing i'  would  state  a very  specific  procedure  for  obtaining  a unique  estimate  r of  z*  in  any  partic- 
ular instance.  However,  a careful  examination  of  existing  rules,  whose  origins  are  always  based 
on  some  rationale,  shows  that  such  a goal  is  difficult  to  attain  in  every  case.  First,  the  pro- 
cedure itself  is  not  always  unambiguously  defined;  sometimes,  certain  parameters  are  left  to  the 
investigator  to  select.  Second,  even  when  these  parameters  are  selected,  the  application  of  the 
rule  does  not  necessarily  result  in  a unique  estimate  — i.e  , certain  judgmental  aspects  remain, 
so  that  two  people  looking  at  the  same  set  of  data  and  carrying  out  identical  procedures  will 
come  up  with  different  estimates  for  l*.  To  remove  these  vagaries  and  attain  unambiguous 
specificity  would  require  substantial  effort  to  pin  these  rules  down  any  further.  As  a matter  of 
fact,  the  performance  characteristics  of  all  of  them  are  so  poor  that  to  make  them  more  explicit 
would  be  a useless  endeavor. 
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. In  ,he  remainder  of  this  paper  we  (1)  discuss  the  criteria  for  goodness  of  a detection  rule, 

(2)  describe  some  rules  that  exist  in  the  literature  and  a slight  modification  of  one,  and,  finally, 

(3)  present  the  results  of  an  empirical  evaluation  study  of  these  rules. 

2.  GOODNESS  CRITERIA 

In  assessing  the  goodness  of  a rule  for  detecting  r*  or  estimating  r *,  there  are  several  desir- 
able characteristics  to  look  for.  These  are  accuracy,  precision,  generality,  low  cost,  and  simpli- 
city. Each  of  these  will  be  discussed  in  turn;  but  first  the  reader  should  be  reminded  again  that 
given  a rule  (including  its  vagaries),  we  are  merely  estimating  a well-defined  number  r*.  Thus 

many  of  the  concepts  associated  with  estimation  theory,  such  as  unbiasedness  and  mean  square 
error,  apply. 

2.1.  Accuracy 

A rule  is  a statement  that  tells  us  how  to  obtain  a r*  Of  course,  this  is  just  one  possible 
value  of  a random  variable  T , i.e.,  T is  an  estimator  of  /*.  Accuracy  will  be  used  as  a measure 
of  location.  Thus,  it  seems  that  an  appropriate  definition  of  accuracy,  a,  would  be 

a _ 

i*  ‘ 

If  this  ratio  is  close  to  one,  then  we  say  the  detector  is  accurate;  if  it  is  greater  than  one  it 
implies  a positive  bias,  and  less  than  one,  a negative  bias. 

2.2  Precision 

Precis'ofl.  A wi|l  be  used  as  a measure  of  variation;  more  specifically,  the  coefficient  of 
variation  of  T*,  namely, 

_ _ ^ar[f*) 

Elf*] 

will  be  defined  as  the  precision  of  the  estimator.  Clearly,  a small  value  is  desirable;  when  p is 
close  to  zero  we  say  that  the  detector  is  precise. 

2.3.  Generality 


This  is  a property  which  means  that  the  rule  performs  well  across  a broad  range  of  sys- 
tems and  a broad  range  of  parameters  within  a system. 

2.4.  Cost 

By  cost  we  mean  the  expense,  in  computer  time,  from  using  a given  detection  rule.  As 
will  be  seen,  there  are  three  factors  which  we  combine  to  arrive  at  a total  cost.  Not  all  factors 
appear  in  every  rule.  These  factors  are 

(i)  computer  time  for  the  algorithm  itself,  i.e.,  its  computational  efficiency, 

(ii)  computer  time  for  collecting  computer  output  data  only  for  a preliminary  estimate  of 
r*,  and  subsequently  discarding  this  data,  and 


* ■■  • % 
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(iii)  computer  time  associated  with  a positively  biased  rule.  Thus,  if  E[T*  1 >>  r*,  then 
on  any  replication  of  the  simulation  experiment,  one  of  two  situations  may  occur,  unbeknownst 
to  the  analyst.  Either  (a)  more  data  is  generated  than  would  be  required  if  one  were  studying 
the  transient  situation,  or  (b)  for  estimating  a steady  state  parameter,  for  instance,  the  use- 
ful data  generated  between  r*and  the  smallest  integer  greater  than  or  equal  to  £[/*]  would  not 
be  used. 

2.5.  Simplicity 

This  is  a characteristic  of  a rule  which  makes  it  accessible  to  the  average  practitioner  of 
large-scale  system  simulation.  A rule  utilizing  abstruse  mathematical  or  statistical  results  is 
nearly  incomprehensible,  and  virtually  useless,  to  the  average  person  who  needs  to  know  how 
to  get  statistically  reliable  results  from  a simulation. 

In  evaluating  any  specific  rule,  its  accuracy,  precision,  and  generality  should  be  considered 
first.  A rule  that  is  not  satisfactory  on  all  three  counts  is  obviously  undesirable,  and  not  worth 
pursuing.  Unless  a rule  is  prohibitively  expensive,  cost  should  be  a relatively  minor  considera- 
tion. However,  it  is  a matter  of  personal  judgement  where  one  balances  budget  constraints  with 
the  necessity  for  a good  predictor.  The  criterion  of  simplicity  is  always  satisfied  in  the  rules  we 
are  considering  in  this  study. 

3.  DETECTION  RULES 

In  this  section  we  describe  the  rules  most  commonly  appearing  in  the  simulation  litera- 
ture. The  results  of  an  empirical  evaluation  study  of  these  rules,  in  terms  of  the  criteria  dis- 
cussed above,  are  presented  later. 

3.1  Rule  1 (Conway  Rule) 

Conway  [1]  says,  "I  usually  truncate  a series  of  measurements  until  the  first  of  the  series 
is  neither  the  maximum  nor  the  minimum  of  the  remaining  set.  I do  not  do  this  for  every  run, 
but  rather  decide  on  a stabilization  period  by  examining  a few  pilot  runs  and  thereafter  delete 
this  same  period  from  the  result  of  each  run." 

Thus,  this  rule  specifies  a priori  the  number  of  exploratory  replications  and  the  number  of 

observations  per  exploratory  run,  denoted  by  e and  I,  respectively.  Then  if  {at,,,  x,7 x,,)  is 

the  set  of  observations  on  the  /,h  exploratory  run,  one  computes 


x,t  - max{x,*,xa  + ,,  . 

• • .xj 

*,*  - min{x(4,xa+,,  ., 

■ ■ . X,,) 

for  A:  — 1,2 land  determines  t,  such  that 

Xu*  ^ Xu*  ^ X/f^ 

occurs  for  the  first  time.  Then  the  estimate  of  z*  is  given  by 

t*  - maxffj.fa f,|. 

A schematic  diagram  of  this  situation  is  shown  in  Figure  2 for  a single  exploratory  run  of  length 

10. 
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Figure  2 — Conway  Rule  applied  lo  a 
single  exploratory  run  of  length  10. 


^ ^ 56  7 89  10 

OBSERVATION  NUMBER 


3.2  Rule  2 (Modified  Conway  Rule) 

we  turn  °n'y  0n*  conJsidered  in  this  PaPer  «»>«  « not  in  the  literature.  In  this  rule, 

■T n2lh«  ? m ■ ar°a  3 Cont,nually  l00k  backwards  to  find  the  first  observation  that 
is  neither  a maximum  or  minimum  of  all  the  previous  observations.  Thus,  the  total  number  of 

method  r0enS  m 'h,S  Pr°Cedr  iS  3 r3nd0m  Variab'e’  in  con,rast  “>  Conway  rul  Ag"n  the 

sTo^nVooTmeSTHPreSPeC,fied  nUmber  °f  eXp'°rat0ry  replications,  e,  each  of  which  produces  a 
stopping  point.  The  maximum  stopping  point  in  this  set  of  stopping  points  is  selected  as  ?. 

A schematic  diagram  of  this  situation  is  shown  in  Figure  3 for  a single  exploratory  run 
The  observation  values  shown  are  the  same  as  those  in  Figure  2.  Note  however  that  the 
simulation  stops,  in  this  case  with  observation  6,  when  the  criterion  is  met 


O OBSERVATION  SELECTED 


Figure  3 — Modified  Conway  rule  applied 
to  a single  exploratory  run. 


1 2 54  56  7 09  to 

OBSERVATION  NUMBER 


3.3.  Rule  3 (Crossings-of-the-Mean  Rule) 

mpanlhi!.rUle’,Teariungjn  Fishman  (Ref-  121  p 275)’  sPec,fies  ‘b“‘  a running  cumulative 

m o he  Z i "V  Ka,Vre  f nerated’  and  ,hat  a count  be  made  of  the  number  of  cross- 
ings of  the  mean,  looking  backwards  to  the  beginning.  When  the  number  of  crossings  reaches 

<’°e  "rlVed  “ * Tl,us-  if  ""  '.I 

"/  “ n > X/+l  < *•  °r  Xj  < *•'  *'+'  > *" 

1 0,  otherwise 


with  y - 1,2,  — 1 , where 


Then  compute 


1 n 

“ T L x/- 

n I- 1 


n-\ 

nn~  Z «/• 

i~\ 
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which  gives  us  the  number  of  times  that  the  series  ....  x„  crosses  the  mean.  One  com- 
putes 11 2.  11 3 stopping  at  fl^when,  for  the  first  time,  the  number  of  crossings  is  greater 

than  or  equal  to  a preassigned  number.  The  intuitive  notion  is  that  the  larger  this  number  is, 
the  greater  is  our  confidence  that  bias  due  to  initial  conditions  has  been  resolved.  This  seems 
reasonable.  For  example,  if  one  were  to  start  the  system  out  empty,  one  would  expect  the 
cumulative  average,  x,„  to  increase  and  level  out  to  its  equilibrium  value.  When  this  happens, 
the  individual  x,'s,  measured  in  the  near  equilibrium  situation,  would  be  sprinkled  on  either  side 
of  the  equilibrium  value  and  thus  contribute  to  an  increasing  ft„.  However,  during  the  early 
stages,  the  individual  x/s  would  fall  above  this  increasing  cumulative  mean,  and  therefore  add 
nothing  to  the  fl„.  A schematic  of  the  situation  is  shown  in  Figure  4. 


Figure  4 — Crossings  of  the  mean 


3.4  Rule  4 (Cumulative-Mean  Rule) 

The  procedure  described  by  Gordon  (Ref.  [3],  p.  285)  for  this  rule  is  to  specify,  a priori, 
the  number  of  exploratory  runs,  the  number  of  observations  in  each  run  and  the  initial  condi- 
tion for  each  that  is  held  constant  for  each  exploratory  run,  then  to  plot  the  grand  cumulative 
mean  over  all  exploratory  runs  and  observations  and  select  the  observation  number  which 
appears  to  be  stable. 


3.5  Rule  5 (Gordon  Rule)  (Ref.  [3],  p.  285) 

For  a very  large  class  of  processes,  namely  ones  for  which  covariance  stationarity  obtains 
and  where 

"•  “ £ R> 

is  finite,  and  where  covariance  of  lag  y,  it  can  be  shown  (Ref.  [3]  p.  281)  that 

VarUJ  - — + of-). 

n n 

where  o(— ) denotes  terms  of  order  higher  than  1/n.  However,  suppose  there  exists  an  initial 
n 

bias.  Label  the  random  variables 

Xn‘+H' 

where  n * corresponds  to  the  point  where  near  equilibrium  is  obtained  and  n is  the  number  of 
additional  observations  beyond  that  point.  Define 

1 


n + n i-t 


1 a* 
Xn%  ” . 

n -I 
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and 


where 


Then,  it  can  be  shown  that 


— 1 " 

Xn  - - £ X,. 

n / -T 


Xi-Xn.+i  J-  1.2 n. 


Varf*„.+n] 


n*  + n 


w . , 1 , 

— + o(— ). 
n n 


Thus,  after  a sufficiently  large  number  of  observations  beyond  «*,  the  variance  of  X„-+n 
behaves  like  the  variance  of  X„  which  has  no  initial  bias.  The  similarity  of  behavior  is  in  the 
sense  that  the  variance  consists  of  a term  of  order  \/n  plus  a term  of  order  higher  than  1 In. 


The  idea,  proposed  by  Gordon  requires  a priori  specification  of  the  number  of  exploratory 
runs,  e,  the  number  of  observations  to  be  made  in  each  exploratory  run,  £,  and  the  initial  condi- 
tion which  is  held  constant  for  each  exploratory  run,  x0.  Denoting  the  mean  value  of  the  j,h 
exploratory  run  by  xi  n , one  may  estimate  Var  l3f„]  by 

sHn)  “ TTT  £ ~ 

e 1 /-i 

where  e is  the  number  of  exploratory  runs, 

N — ne, 

and 


xti  - /,h  observatioin  of  the  j'h  exploratory  run. 

As  indicated  above,  in  the  absence  of  initial  bias,  s1(n)  can  be  expected  to  be  inversely 
proportional  to  n or  s(n ) inversely  proportional  to  Vn  for  larger  values  of  n;  however,  in  the 
presence  of  initial  bias,  such  as  our  constant  initial  condition,  this  behavior  will  manifest  itseft 
only  after  the  effects  of  initial  conditions  are  eliminated.  Gordon  suggests  plotting  s(n ) versus 
n on  log-log  paper  and  finding  the  point  on  the  plot  where  it  steadies  into  a straight  line  with  a 
negative  slope  of  1/2. 

3.6  Comment 

A very  important  observation  here  is  that  Rule  3 wastes  no  data.  Thus,  in  practice,  the 
run  for  determining  f*  would  be  included  as  part  of  the  simulation,  so  that  in  carrying  out  the 
experiment,  when  a point  in  time  is  declared  as  near  equilibrium,  the  experimenter  may  then 
stop  (if,  say,  he  is  conducting  a study  of  transient  characteristics),  or  he  may  continue  from  r 
on,  to  collect  data  while  the  system  is  near  steady  state.  The  point  is  that  there  is  no  separation 
of  experiments;  i.e.,  one  set  to  determine  a near-equilibrium  point  and  a second  set  of  data- 
collecting  experiments.  This  is  not  the  case  for  rules  1,  2,  4,  and  5,  as  they  are  not  an  integral 
part  of  the  simulation  model.  Rather,  these  rules  require  that  an  initial  set  of  experiments  be 
conducted  to  specifications  laid  down  by  the  rule,  the  sole  purpose  being  to  provide  data  for 
estimating  an  appropriate  stopping  point.  The  actual  data-collecting  experiments  require  addi- 
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tional  replications  of  the  simulation.  All  the  data  generated  for  establishing  the  near- 
equilibrium  point  are  essentially  wasted. 

4.  EMPIRICAL  EVALUATION  STUDY 

As  stated  earlier,  the  criterion  of  simplicity  has  been  met  by  all  live  rules  described  above. 
Each  of  them  would  be  comprehensible  to  any  simulator  with  a modest  background  in 
mathematics  and  mathematical  statistics,  and  is  readily  usable.  So  this  point  will  no  longer  be 
addressed. 

The  next  three  criteria  to  be  considered  are  those  of  accuracy,  precision,  and  generality. 
The  obvious  way  to  go  about  doing  this  is  to  look  at  how  the  detection  rule  performs  over  a 
wide  spectrum  of  systems.  We  began  with  one  of  the  best-known  queuing  systems,  namely, 
M/M/1.  If  a rule  performs  well  in  this  instance,  i.e.,  it  is  accurate  and  precise  over  a wide 
range  of  initial  conditions  x0  “ number  of  customers  in  the  system  at  t - 0 and  parameter 

values  p - — , where  A is  equal  to  the  arrival  rate  and  p.  the  service  rate,  respectively,  then  it 
M 

would  be  necessary  to  assess  the  generality  of  the  rule.  These  considerations,  i.e.,  accuracy  and 
precision,  would  have  to  be  studied  with  other  systems.  This  study  is  only  a beginning.  The 
M/M/l  queuing  system  was  chosen  for  the  following  reasons: 

(1)  Queuing  simulations  occur  often. 

(2)  Theoretical  values  of  the  quantities  of  interest  can  be  computed  with  relative  ease. 

(3)  It  is  easy  and  inexpensive  to  simulate. 


In  order  to  carry  out  our  evaluations,  it  is  necessary  to  determine  the  true  t*  for  various 
x0  and  p combinations.  This  means  that  the  expectation  E[X,\  must  be  known  as  a function  of 
t as  well  as  E[X  J.  These  are  basically  analytical  problems;  and,  as  pointed  out  in  the  preced- 
ing paragraph,  one  reason  for  selecting  A// AS/1  is  that  the  analysis  is  tractable. 

The  observed  random  variable  (chosen  from  many  possible  outputs),  referred  to  in  gen- 
eral as  X,  so  far,  is,  for  the  M/M/l  model, 

W,  - waiting  time  in  queue  of  the  rth  arrival  before  being  served. 

The  simulation  was  carried  out  recursively  using 

Wt  - max(0,  W, + S,_,  - A/), 


where 

S,_,-  service  time  of  the  (t-  l)lh  arrival, 

A,  — interarrival  time  of  the  /,h  arrival. 

E[W]  for  an  initial  condition  of  empty  is  given  in  Ref.  [4],  The  expectations  for  nonempty 
conditions  were  developed  in  the  course  of  this  study;  see  Ref.  (6)  for  details. 


RULES  FOR  DETECTING  STEADY  STATE 


519 


The  main  objective  of  the  empircal  study  was  to  estimate  the  accuracy  and  precision  of 

i no  Vrri0US  ufU  ef'  F?r  fUleS  2’  and  3’  we  made  100  '"dependent  estimates  / - 1,2 

1UU,  i or  each  value  of  p and  Xq  considered  and  used 


i ^ 

ioo  fr'' 

a £[7-.] 


as  the  estimate  of  the  accuracy  a,  and 


as  the  estimate  of  precision,  p,  where 


(^ar  [T*] 


ElT*] 


“loo 


„ 1 100 

£l^'- 150,5^ 

Each  of  the  100  estimates  was  determined  by  running  the  simulation  and  selecting  ^in  accor- 
dance with  the  prescription  provided  by  the  rule.  This  procedure,  of  replicating  100  times,  was 
too  expensive  for  rules  4 and  5.  However,  we  are  confident,  based  on  only  one  replication,  that 
our  conclusions  regarding  these  rules  are  correct. 

4.1  Rule  1 (Conway  Rule) 

We  have  followed  the  spirit  of  Conway’s  thought  that  one  should  use  a "few"  exploratory 
replications  and  a small  number  of  observations.  The  specified  number  of  exploratory  replica- 
tions, e , was  always  either  1,  3,  5,  or  10.  The  specified  length  of  each  replication,  /(i.e.,  the 
number  of  customers),  was  varied  as  follows:  4(1)  10(2)  20(5)  30(10)  50(25)  100.  A wide 
range  of  parameters  p and  Xq  was  tested,  as  shown  in  Table  1 . In  general,  it  was  possible  to 
make  100  replications  for  each  p,  Xq,  e,  l combination.  However,  in  some  situations,  especially 
for  low  / and  p values,  a replication  may  produce  a sequence  of  waiting  times  for  which  the 
Conway  criterion  is  not  met  , i.e.,  there  exists  no  observation  such  that  it  is  neither  a minimum 
or  a maximum  of  all  succeeding  observations.  For  example,  in  the  case  of 
p — 0.1,  Xq  — 0,  e — 1,  and  1—5,  only  one  out  of  the  100  replications  was  successful  in  pro- 
ducing a f* . Table  2 shows  the  number  of  successful  replications  out  of  100,  as  a function  of 
the  exploratory  run  length/,  for  the  conditions  specified  previously.  Obviously,  the  estimates 
of  accuracy  and  precision  are  based  only  on  the  subset  of  successful  runs. 

TABLE  1 — MIMI1  Queuing 
Parameters  Used  to  Test 
the  Conway  Rule 


\*0 

P 0 

5 

10 

0.1  x 

X 

0.5  x 

X 

0.7  x 

X 

0.9  x 
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TABLE  2 — Number  of  Successful 
Replications  out  of  100  Using 
the  Conway  Rule  with  p =0.1, 
x0  = 0,  and  e = 1 


Number  of  Successful 
Replications 

5 

1 

10 

7 

15 

18 

20 

27 

25 

43 

30 

55 

40 

72 

50 

82 

75 

97 

100 

98 

The  Conway  rule  resulted  in  a very  poor  set  of  performance  characteristics.  Figures  5a,  b, 
c,  and  d show  plots  of  accuracy  for  p « 0.1,  0.5.  0.7,  and  0.9,  respectively,  and  various  values 
of  the  other  parameters.  These  plots  are  all  for  e — 0.10;  however,  the  results  are  essentially 
identical  for  e — 0.05.  They  show  that  for  low  p values  this  rule  overestimates  f*,  and  for  high 
p values  it  grossly  underestimates  t*.  Thus,  it  fails  on  the  accuracy  criterion. 

Note  that  e = 1 does  not  appear  on  Figure  5a.  This  was  because  there  were  not  enough 
successful  replications  to  get  good  estimates.  This  is  also  the  reason  why  the  left  endpoint  is 
not  the  same  for  all  curves.  The  plots  begin  only  when  l is  sufficiently  large  to  produce  enough 
replications  for  a good  estimate. 

Similar  plots  for  estimates  of  precision  were  also  obtained.  Figures  6a  and  b,  for  p — 0.5, 
are  typical  of  these  results  — roughly  that  0.3  < p < 0.6  for  x0  - 0,  and  0.4  ^ p ^ 0.8  for  the 
nonempty  starting  case,  i.e.,  when  x0  ” 5,  10.  and  25,  for  p - 0.1,  0.5,  0.7,  and  0.9,  respec- 
tively. 


4.2  Rule  2 (Modified  Conway  Rule) 


In  the  original  form  of  the  Conway  Rule,  the  length  of  the  exploratory  runs,  /,  must  be 
specified  in  advance,  and  one  looks  ahead  only  in  order  to  make  a determination  of  whether  or 
not  the  criterion  is  met.  In  the  Modified  Conway  Rule,  since  one  only  looks  back  for  making 
the  determination,  there  is  no  need  to  specify  an  exploratory  run  length;  i.e.,  the  exploratory 
run  continues  until  the  criterion  is  met  and  an  estimate  r*is  acquired.  Thus,  the  only  parame- 
ter in  this  version  of  the  rule  is  the  number  of  exploratory  runs,  e.  This  also  makes  presenta- 
tion of  the  data  substantially  simpler. 

The  performance  in  this  case  was  very  poor  also.  Figures  7a,  b,  c,  and  d summarize  the 
accuracy  results  for  c - 0.10;  the  results  for  < - 0.05  are  essentially  the  same.  In  these  particu- 
lar sets  of  runs,  an  additional  value  of  xn  was  run  for  each  value  of  p.  This  intermediate  value 
was  in  some  cases  selected  to  be  very  close  to  the  steady-state  number  in  the  system,  which  is 
given  by  p/(l  - p).  Thus,  for  example,  in  Figure  7d,  the  additional  value  is  xQ  - 10  and  the 
steady-state  value  is  9.  The  Modified  Conway  Rule  badly  underestimates  t*  in  virtually  all 
cases,  except  for  the  intermediate  x0  values  at  p - 0.5  and  p - 0.7. 
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4.3  Rule  3 (Crossings-of-the-Mean  Rule) 

In  testing  this  rule,  the  only  open  parameter  we  considered  is  the  number  of  crossings  to 
be  attained  before  declaring  that  near-equilibrium  had  been  reached.  The  range  of  system 
parameters  was  the  same  as  that  used  in  the  Conway  Rule  (see  Table  1).  Figures  9a,  b,  c,  and 
d display  accuracy  versus  the  number  of  crossings  for  p - 0.1. 0.5, 0.7,  and  0.9,  respectively.  It 
is  clear  from  these  figures  that  the  rule  is  very  conservative  for  low  p values,  i.e.,  it  overesti- 
mates the  i*  and  provides  accuracies  much  greater  than  one,  no  matter  what  criterion  is  selected 
for  the  number  of  crossings.  As  p increases  it  becomes  less  conservative;  in  fact,  for  p — 0.9 
its  accuracy  is  less  than  one  for  crossings  below  25.  Nevertheless,  it  appears  that  a crossings 
criterion  of  25  would  work  uniformly  across  all  system  parameters  in  the  sense  that  it  provides 
an  estimate  r*  which,  on  the  average,  is  greater  than  r*for  all  conditions  tested.  Thus,  the  sys- 
tem is  closer  than  expected  to  the  equilibrium  condition. 


(c)  (d) 


F.gure  9 — Accuracy  characteristics  of 
Crossings-of-the-Mean  Rule. 


The  fact  that  this  rule  overestimates  /*  is  a serious  problem  in  terms  of  the  number  of 
excess  observations  required  beyond  the  true  near-equilibrium  point.  To  illustrate  this  point, 
consider  Table  3,  which  shows  the  pairs  for  the  cases  tested.  The  first  coordinate  of  the 
pair  is  the  true  t*  value  for  c - 0.1  and  the  second  coordinate  is  the  estimated  accuracy,  a. 
Thus,  for  example,  the  excess  number  of  observations  in  the  case  of  p - 0.1  and  x0  - 0 is 
(3)  (55  - 1)  - 162,  and  this  is  also  the  largest  excess.  In  all  situations  of  excess,  the  actual 
attained  value  of  < is  nearly  0,  i.e.,  the  process  is  essentially  in  equilibrium. 

Now,  if  we  wanted  to  be  within  an  c - 0.05  of  p.„,  the  (r*,a)  would  be  as  shown  in  Table 
4.  The  maximum  excess  is  the  same  value  of  162  but  the  other  excesses  have  been  reduced  at 
the  expense  of  an  accuracy  of  only  0.7  for  p - 0.9. 
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TABLE  3 — ( t*,a ) for  t — 0.1  and 
a 25-Crossing  Criterion 


(t*.  a) 

I I I 


xo-0  x0-5  xo-10 


0.1 

(3,55) 

0.5 

(10,12) 

0.7 

(29,6) 

0.9 

(272,1) 

(58,3.5) 


(413,1) 


TABLE  4 — 0*.  a)  for  t — 0.05  and  a 
2 5 -Crossing  Criterion 


O' 

*,  a) 

x0-0 

1 

x0-5 

xo-10 

xo-25 

0.1 

(3,55) 

0.5 

(14,7) 

0.7 

(42,4) 

0.9 

(411,0.7) 

(6,28) 


(20,6) 


(73,2.8) 


(552,0.7) 


Table  5 shows  the  excess  number  of  observations  for  « - 0.1  and  0.05  where  the  figures 
above  and  below  the  diagonals  are  for  t — 0.1  and  0.05,  respectively.  A negative  excess 
means,  of  course,  that  additional  observations  are  required  to  achieve  the  desired  near- 
equilibrium level. 


TABLE  5 — Excess  Number  of  Observations  for  t — 0.1 
(Above  Diagonal)  and  0.05  (Below  Diagonal)  for  a 
25-Crossing  Criterion 


If  a 30-crossing  criterion  is  specified,  the  corresponding  results  are  shown  in  Tables  6,  7, 
and  8.  In  this  case,  the  accuracies  are  always  greater  than  one  and  result  in  an  increase  in  the 
excess  number  of  observations. 
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TABLE  6 — (/*.  a)  for  t — 0.1  and  a 
30-Crossing  Criterion 


1 

0* 



a ) 

*o  “ 0 

*o“  10 

*o-  25 

0.1 

(3,65) 

(5,40) 

0.5 

(10,14) 

(16,10) 

0.7 

(29,7) 

(58,4.5) 

0.9 

(272,1.5) 

(413,1.5) 

TABLE  7 — ( t *,  a)  for  e « 0.05  and  a 
30-Crossing  Criterion 


P 

0*.  a) 

WH1 

Em 

ES9 

x„-25 

0.1 

(3,65) 

(6,33) 

0.5 

(14,10) 

(20,8) 

0.7 

(42,4.8) 

(73,3.6) 

0.9 

(411,1) 

(552,1.1) 

TABLE  8 — Excess  Number  of  observations  for  t — 0.1 
(Above  Diagonal)  and  0.05  (Below  Diagonal) 
for  a 30- Crossing  Criterion 


Curves  similar  to  the  accuracy  ones  were  developed  for  precision.  A sample  of  one  of 
these  is  shown  in  Figure  10.  In  general,  precision  improves  with  decreasing  p and  increasing 
values  of  the  crossing  criterion.  A summary  of  precision  estimates  for  all  the  cases  tested  is 
shown  in  Table  9 for  crossings’  criteria  of  25  (above  the  diagonal)  and  30  (below  the  diagonal). 
These  values  of  precision  would  appear  to  be  intolerable,  in  view  of  the  fact  that  the  t*  values, 
say  for  p -*  0.9,  are  between  300  and  500. 

4.4  Rale  4 (Cumulative-Mean  Rule) 

This  rule  was  tested  using  10,000  arrivals.  Table  10  shows  the  number  of  replications 
(NR)  and  the  utilization  ratios,  p,  which  were  used.  The  initial  conditions,  jtq,  were  0 and  5 
for  p ■■  0.5  and  0 and  25  for  p - 0.9. 
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TABLE  9 — Precision  Values  for  Crossings  ’ Criteria  of 
25  (Above  Diagonal)  and  30  (Below  Diagonal) 


TABLE  10—  Utilization  Ratios  and  Number  of  Replications 
Used  in  Testing  the  Cumulative  Mean  Rule 


p 

Number  of  Replicatons  (NR) 

0.5 

10 

50 

100 

200 

0.9 

10 

50 

100 

200 

Typical  plots  of  the  cumulative  mean  waiting-time-in-queue  versus  n (the  customer 
number),  for  the  empty  initial  condition,  are  shown  in  Figures  11a  and  b.  In  addition,  for  p “ 
0.5  and  NR  — 200,  a stationary  simulation  was  performed,  i.e.,  one  in  which  the  number  of 
customers  in  the  system,  when  the  simulation  is  begun,  is  a random  variable  with  the  steady- 
state  number  in  system  distribution.  The  result  is  shown  in  Figure  11c.  The  true  steady-state 
mean  delay  in  queue,  in  these  figures,  is  one  for  p - 0.5  and  nine  for  p — 0.9. 

An  inspection  of  these  figures  reveals  (1)  that  it  is  difficult  to  determine,  from  a small 
number  of  exploratory  runs,  when  stabilization  has  occurred,  (2)  that  a large  number  of  obser- 
vations per  run  are  required  to  make  any  determination  of  stability,  and  (3)  that  t*  is  grossly 
biased  positively,  even  in  the  stationary  case  of  Figure  11c. 


The  conclusion  seems  obvious;  the  cumulative  mean  is  a very  poor  method  for  estimating  t* 
and  we,  therefore,  do  not  intend  to  consider  it  any  further,  e.g.,  no  estimate  of  precision  has 
been  obtained  since  it  would  require  an  inordinate  amount  of  computer  time.  As  pointed  out 
by  Law  [5],  it  appears  to  be  a technique  that  would  determine  the  value  of  n for  which 
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4.5  Rule  5 (Gordon  Rule) 

This  rule  was  tested  using  the  same  conditions  as  Rule  4 (Cumulative-Mean  Rule)  and,  in 
fact,  the  same  data.  Corresponding  to  Figures  11a,  b,  and  c are  Figures  12a,  b,  and  c.  The 
straight  lines  on  these  figures  slope  downward  at  the  rate  of  1 in  2.  Again,  it  is  seen  that  a large 
number  of  exploratory  runs  and  observations  per  run  are  required  in  order  to  carry  out  the  pro- 
cedure. Also,  t * is  extremely  biased  positively,  even  in  the  stationary  case  (Figure  12c).  The 
positive  bias  is  to  be  expected,  since  even  in  the  stationary  case  Var[T„]  is  inversely  propor- 
tional to  n only  for  large  values  of  n.  Thus,  for  p — 0.5,  t*  from  Figure  12a  would  be  taken  as 
600;  yet  the  t * values  would  be  10  (Table  3)  and  14  (Table  4)  for  t — 0.1  and  0.05,  respec- 
tively. Similarly,  for  p — 0.9,  t*  would  be  taken  as  1200  (Figure  12b)  and  the  t*  values  for  « 
— 0.1  and  0.05  would  be  272  (Table  3)  and  411  (Table  4),  respectively.  We  conclude,  again, 
that  this  is  a poor  method  for  estimating  t\  and  shall  not  pursue  it  any  further.  As  in  the  case 
of  Rule  4,  no  effort  was  made  to  determine  its  precision,  characteristics. 


5.  SUMMARY 

In  this  paper  we  have  developed  a comprehensive  framework  within  which  may  be  pur- 
sued a study  of  the  problem  of  the  initial  transient  with  respect  to  mean  value.  In  addition, 
four  rules  quoted  frequently  in  the  literature,  plus  a natural  variant  of  one  of  them,  have  been 
evaluated  in  the  M/M/\  situation.  The  principal  conclusion  is  that  none  of  the  five  rules  is 
satisfactory  and  that  they  should  not  be  recommended  for  use  by  practitioners. 


In  summary,  the  Conway  and  Modified  Conway  Rules  badly  underestimate  t * they  are 
costly  in  the  sense  of  wasting  data,  and  there  exists  no  procedure  for  determining  either  the 
number  or  length  of  exploratory  runs. 
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(a) 


(b) 


Figure  12  — Characteristics 
of  the  Gordon  Rule. 


(c) 


The  Crossings-of-the-Mean  Rule  when  applied  to  high  p values,  dictates  a crossing  crit- 
gerion  of  about  30.  But  this  causes  a large  positive  bias  for  the  lower  p values,  and  hence,  is 
wasteful  of  data.  Also,  its  precision,  especially  at  the  higher  p values,  is  unacceptable. 

Both  the  Cumulative-Mean  Rule  and  the  Gordon  Rule  are  badly  biased  positively.  They 
have  too  many  judgmental  factors,  such  as  the  number  and  length  of  replications.  If  these  are 
not  selected  to  be  very  large  initially,  then  there  is  the  chance  that  all  the  data  generated  cannot 
be  used,  and  the  whole  procedure  must  be  started  anew  from  the  beginning;  unless,  of  course, 
a snapshot  of  the  system  at  the  end  of  each  replication  has  been  preserved,  so  that  additional 
observations  can  be  added  to  each  run  without  starting  all  over.  It  might  be  argued  that  both 
these  techniques,  when  applied  with  a sufficient  number  of  runs  and  observations  per  run, 
automatically  provide  an  estimate  of  the  mean.  In  the  case  of  the  Cumulative-Mean  Rule,  the 
stabilization  value  would  be  that  estimate;  and  in  the  case  of  the  Gordon  Rule,  an  estimate 
could  be  provided  by  using  the  cumulative  mean  of  all  the  observations  beyond  the  truncation 
point.  The  point  is  that  both  are  extremely  costly  — substantially  more  data  have  been  gen- 
erated than  are  needed  to  achieve  the  goal. 
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ABSTRACT 

The  problem  considered  is  lo  locale  one  or  more  new  facililies  relative  to  a 
number  of  existing  facilities  when  both  the  locations  of  the  existing  facilities, 
the  weights  between  new  facilities,  and  (he  weights  between  new  and  existing 
facilities  are  random  variables.  The  new  facililies  are  to  be  located  such  that 
expected  distance  traveled  is  minimized.  Euclidean  distance  measure  is  con- 
sidered; both  unconstrained  and  chance-constrained  formulations  are  treated. 

1.  INTRODUCTION 

To  date,  the  study  of  location  problems  has  been  restricted  primarily  to  deterministic  for- 
mulations. In  this  paper  the  effect  of  random  variation  on  the  location  decision  will  be  studied. 
Probabilistic  formulations  of  the  multifacility  Weber  problem  are  developed  and  solution  pro- 
cedures are  obtained.  Applications  of  the  various  formulations  are  cited  to  motivate  an  under- 
standing of  the  contexts  in  which  the  formulations  apply. 

The  elements  of  a facilities-location  problem  which  are  treated  as  random  variables  are  the 
locations  of  the  existing  facilities  and  the  amount  of  interaction  between  new  and  existing  facili- 
ties. As  an  illustration  of  a location  problem  in  which  random  variation  can  exist,  consider  the 
location  of  a maintenance  department  for  material-handling  equipment  in  an  industrial  plant. 
Maintenance  performed  is  of  two  types,  scheduled  and  unscheduled.  Unscheduled  maintenance 
arises  when  the  material-handling  equipment  fails  and  a repairman  is  dispatched  to  the  job  site 
to  perform  the  necessary  repairs.  The  location  of  the  equipment  when  it  fails  is  a random  vari- 
able. Additionally,  the  number  of  times  a particular  piece  of  equipment  fails  during  a year  is  a 
random  variable.  The  determination  of  the  location  of  the  maintenance  department  based  on 
the  random  variation  involved  is  typical  of  the  location  problems  considered. 

The  deterministic  formulation  of  the  multifacility  Weber  probjem  is  given  by 
PI.  minimize /(AT, AT,)-  £ ^|ATy-Af*|+  £ 

X)  !</<*<«  /-I  /-I 
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where 

location  of  new  facility  j,  j— 1 », 

location  of  existing  facility  i,  i—l m, 

number  of  trips  per  unit  time  between  new  facilities  j and 
k,  for  all  j < k, 

number  of  trips  per  unit  time  between  new  facility  j and 
existing  facility  /,  for  all  j,  i. 

Euclidean  distance  between  the  points  Xj  and  P„  and 
total  distance  traveled  per  unit  time  as  a function  of 

*1 x„. 

In  PI,  the  objective  is  to  determine  the  locations  of  the  new  facilities  in  order  to  minimize 
total  distance  traveled  per  unit  time.  In  the  subsequent  discussion,  it  is  assumed  that  all  new 
facilities  are  chained  [4], 

The  treatment  of  the  probabilistic  variation  of  PI  includes  the  possibility  of  P„  V,k , and 
Wn  being  random  variables.  Two  types  of  probabilistic  problems  are  studied.  In  the  first  case 
it  is  assumed  that,  for  a given  realization  of  P„  a realization  of  Wh  occurs.  Once  the  location 
of  existing  facility  / is  known,  all  subsequent  trips  between  new  facility  j and  existing  facility  / 
will  share  the  same  distance  \Xj  - P,\ , and  the  weight  attached  to  the  trip  will  be  W From  a 
probability  point  of  view,  the  distance  traveled  between  new  facility  j and  existing  facility  i is 
expressed  as  a product  of  the  random  variables  W ,,  and  | X,  — P, |.  In  the  second  case  con- 
sidered, for  each  trip  included  in  W the  distance  between  new  facility  j and  existing  facility  / 
can  be  different.  There  are  trips  during  the  planning  horizon  under  investigation,  and 
existing  facility  / changes  its  location  during  this  planning  horizon  independent  of  the  weight 
W/„.  For  convenience,  let  P,„  denote  the  location  of  existing  facility  / on  trip  h.  Thus,  on  trip 
h , the  distance  traveled  is  determined  from  the  value  of  the  random  variable  P,h.  The  "weight" 
or  number  of  trips  per  unit  time  is  considered  to  be  independent  of  the  location  of  each  exist- 
ing facility.  Thus,  the  distance  traveled  can  be  represented  as  a random  sum  of  random  vari- 
ables. 


Xj 

P, 


Jk 


\xk-p,\ 
fix,.  . 


X„)  - 


To  illustrate  the  first  of  the  two  cases  considered,  let  us  suppose  new  warehouses  are  to  be 
located  across  the  country.  The  sources  of  goods  shipped  to  the  warehouses  and  the  destina- 
tions of  goods  shipped  from  the  warehouses  are  not  known  a priori.  However,  after  the 
warehouses  become  operational,  the  locations  of  suppliers  and  customers  will  become  known. 
The  number  of  shipments  per  month  from  the  suppliers  to  the  warehouses  and  from  the 
warehouses  to  customers  is  not  known  exactly,  but  can  be  expressed  in  the  form  of  a probabil- 
ity distribution.  Since  all  shipments  from  supplier  / to  warehouse  j will  be  from  the  point  P„ 
once  the  value  of  P,  becomes  known  the  location  problem  can  be  formulated  as  a function  of 
the  sum  of  the  products  of  the  random  variables  \X,~  P,\  and  IV,,. 

As  an  illustration  of  the  second  case,  let  us  suppose  a military  hospital  is  to  be  located  to 
provide  medical  treatment  for  personnel  wounded  in  combat.  Patients  are  brought  from  the 
area  to  the  hospital  in  helicopters.  There  are  m combat  areas,  and  the  number  of  helicoter  trips 
to  and  from  combat  area  / is  a random  variable  Wr  The  location  of  a wounded  soldier  in  com- 
bat area  / is  a random  variable  denoted  by  P,.  Thus,  each  of  the  W,  trips  can  be  to  a different 
location  in  combat  area  i.  In  this  case,  the  location  problem  is  formulated  as  a random  sum  of 
random  variables. 

Research  on  probabilistic  formulations  of  the  Weber  problem  has  included  the  treatment 
of  the  single-facility  problem  by  Cooper  [6],  who  assumed  P is  distributed  as  a bivariate  normal 
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probability  density  function,  W,  is  deterministic,  and  distance  is  Euclidean.  All  random  vari- 
ables he  assumed  to  be  independent.  Cooper  employed  a convergent,  linear  iteration  technique 
to  minimize  expected  cost  per  unit  time.  The  convergence  of  the  algorithm  was  established  by 
Katz  and  Cooper  [18].  The  same  iterative  algorithm  was  employed  subsequently  by  Katz  and 
Cooper  [19]  in  treating  both  exponential  and  symmetrical  exponential  distributions  as  the 
probability-density  function  for  P,. 

Hurter  and  Prawda  [16]  solved  the  Euclidean,  single-facility  location  problem  when  the 
quantities  of  service  demanded  are  independent  random  variables.  They  formulated  the  prob- 
lem as  a chance-constrained  programming  problem,  but  the  constraints  were  used  to  bound  W, 
instead  of  bounding  the  cost  of  transportation,  which  is  a function  of  the  distance.  In  their 
analysis,  the  locations  of  the  existing  facilities  are  assumed  to  be  deterministic  when  the  proba- 
bilistic problem  is  transformed  to  a deterministic  equivalent  problem.  Hurter  and  Prawda 
showed  that  any  existing  algorithm  for  solving  the  deterministic  single-facility  problem  can  be 
used  to  solve  their  chance-constrained  problem. 

The  only  previous  research  on  a probabilistic  formulation  of  the  multifacility  Weber  prob- 
lem appears  to  be  the  chance  constrained  formulation  of  Seppala  [28],  In  his  model,  Vjk  and 
W n are  treated  as  random  variables,  but  P,  is  assumed  to  be  deterministic  and  Euclidean  dis- 
tances are  employed.  Seppala  employs  a fractile  criterion  rather  than  an  expected-value  cri- 
terion. Using  the  approach  developed  by  Charnes  and  Cooper  [5]  to  convert  the  chance  con- 
straint to  its  deterministic  equivalent,  Seppala  obtains  a nonlinear  objective  function.  To  solve 
his  model,  the  CHAPS  algorithm  developed  in  [27]  is  used  to  convert  the  nonlinear  objective 
function  to  a linear  objective  function  augmented  by  some  nonlinear  convex  constraints.  A 
linear  approximation  algorithm  similar  to  MAP,  introduced  by  Griffith  and  Steward  [13],  is 
employed  to  solve  the  resulting  formulation. 

2.  PRINCIPLES  OF  CHOICE 

In  modeling  a real-world  decision  problem,  Morris  [21]  suggests  thr.t  three  alternatives  are 
available.  The  problem  can  be  modeled  as  a decision  under  assumed  certainty,  a decision  under 
risk,  or  a decision  under  uncertainty.  The  research  to  date  on  location  problems  has  concen- 
trated on  modeling  the  problem  as  a decision  under  assumed  certainty.  Thus,  deterministic 
approaches  were  taken,  where  either  the  total  travel  cost  was  minimized  (minisum  criterion),  or 
the  maximum  travel  cost  was  minimized  (minimax  criterion).  The  present  research  effort  con- 
centrates only  on  modeling  location  problems  as  decisions  under  risk. 

In  a decision  under  risk  it  is  assumed  that  the  probability  distributions  are  known  for  all 
random  variables.  Also,  a number  of  alternate  principles  of  choice  are  possible.  Sengupta  and 
Portillo-Cambell  [26]  suggest  four  possible  optimization  criteria  under  conditions  of  risk: 

(a)  expected-value  criterion 

(b)  portfolio  criterion 

(c)  aspiration-level  criterion 

(d)  fractile  criterion. 

The  expected-value  criterion  involves  the  determination  of  the  location  of  the  new  facili- 
ties such  that  an  appropriately  defined  expected-cost  function  is  minimized.  The  portfolio  cri- 
terion seeks  the  location  which  minimizes  the  variance  of  cost,  subject  to  a constraint  on  the 
expected  cost  produced.  An  aspiration-level  criterion  is  used  when  the  facilities  are  to  be 
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located  such  that  the  probability  of  cost  being  less  than  some  specified  value  is  maximized.  The 
fractile  criterion  is  difficult  to  express  verbally;  it  involves  the  location  of  the  new  facilities  so 
that  one  minimizes  the  value  on  the  cumulative  distribution  function  of  total  cost  which 
represents  the  acceptable  probability  of  cost  exceeding  that  value.  Mathematically,  the  fractile 
criterion  can  be  stated  as 


minimize  2 

subject  to:  Pr  (/(x)  < z)  ^ a, 

where  fix)  is  the  total  cost  resulting  from  x,  the  vector  of  coordinate  locations  on  the  new 
facilities,  and  z is  the  cost  below  which  total  cost  occurs  with  at  least  a probability  of  a.  The 
deterministic  equivalent  form  of  the  fractile  criterion  (see  Ref.  [5])  is  written  as 

minimize  £(/(x)]  + K SD[/(x)l, 

where  £(...)  and  SD[...)  are  the  expected  total  cost  and  standard  deviation,  respectively  and 
k is  some  number  of  standard  deviations  derived  from  the  probability  distribution  for  a given 
value  of  a and  z. 


Depending  upon  the  situation  considered  and  the  preferences  of  the  decision  maker,  addi- 
tional constraints  can  be  added  to  the  four  basic  principles  of  choice.  As  an  illustration,  one 
might  wish  to  minimize  the  total  expected  distance  traveled  between  all  facilities,  with  the  res- 
triction that  the  probability  of  the  total  distance  exceeding  100  miles  must  be  less  than  0.10. 


The  aspiration-level  and  fractile  criteria  require  that  the  probability  distribution  for  total 
cost  be  known,  whereas  the  expected-value  and  portfolio  criteria  require  a knowledge  of  at 
most  the  first  two  moments  of  the  distribution  of  total  cost.  Consequently,  in  order  to  provide 
sufficient  information  to  model  a specific  location  problem  using  the  appropriate  principle  of 
choice,  the  probability  distribution  for  total  cost  will  be  developed  for  the  situations  considered. 
Then  the  first  two  moments  will  be  derived. 

Geoflfrion  (121  discussed  the  above  criteria  and  the  relationships  between  them.  Sengupta 
and  Portillo-Campbell  (26)  and  Hazell  (15)  supported  Geoffrion’s  observations  with  computa- 
tional results.  For  a critique  of  the  effectiveness  and  the  pros  and  cons  of  the  described  cri- 
teria, see  Hazell  (15). 

In  the  sequel,  two  criteria  are  considered.  The  first  is  an  analogue  of  the  minisum  cri- 
terion for  the  deterministic  case,  where  the  expected  total  random  cost  is  minimized.  Using  the 
expected-value  criterion  naturally  has  a higher  risk  than  using  other  criteria,  but  if  we  assume 
that  the  variance  of  each  random  location  is  small  and  the  correlations  between  the  existing 
locations  are  relatively  small,  then  the  expected  total  cost  over  a large  time  horizon  gives  the 
decision  maker  reasonable  low-risk  information.  If  the  decision  maker  is  concerned  about  a 
realization  near  the  tail  of  the  probability  distribution,  e g.,  in  the  case  of  the  emergency- 
facilities  location  problem,  where  the  maximum  random  distance  is  of  interest,  other  criteria 
may  be  used  to  reduce  the  risk.  The  expected-value  criterion  may  be  considered  a special  case 
of  the  fractile  criterion,  when  * - 0. 


The  second  criterion  considered  in  this  paper  is  analogous  to  the  fractile  criterion.  Basi- 
cally, it  involves  minimizing  expected  total  cost  subject  to  probabilistic  constraints  on  acceptable 
levels  of  risk,  i.e.,  a chance-constrained  programming.  The  fractile  criterion  is  an  analogue  of 
the  minimax  criterion  for  the  deterministic  case.  Fried  (11]  compared  the  chance-constrained 
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programming  with  both  the  fractile  and  the  portfolio  models;  he  demonstrated  the  stability  of 
the  chance-constrained  model  relative  to  the  other  two  models.  Also  it  is  a consistent  method 
of  treating  the  utility  choices  involved  in  trading-off  risk. 

In  Section  3 expected  total  cost  criteria  are  used  as  an  unconstrained  probabilistic  prob- 
lem, and  in  Section  4 chance-constrained  programming  is  used  in  a constrained  probabilistic 
problem.  The  chance-constrained  model  will  provide  a good  tool  for  sensitivity  analysis  on  the 
acceptable  level  of  risk. 


3,  UNCONSTRAINED  PROBABILISTIC  FORMULATION* 

In  this  section,  the  two  problems  discussed  above  are  formulated  mathematically.  The 
first  problem  is  identified  as  the  case  when  the  expected  cost  is  the  product  of  the  random  vari- 
ables P,  and  W ^ the  second  problem  is  associated  with  the  case  when  the  expected  cost  is  a 
random  sum  of  random  variables. 

For  the  case  of  the  product  of  the  random  variables,  the  expected  total  cost  function  is 
given  by 

P2.  minimize  E [/(*, X„))  - E\  £ 

xi  li<y<*<» 

+ i i "Ax,  - /»i| 

/-i  /-i  J 

“ I ElVlk)\Xj-Xk\ 

l<  / </r  < a 

+ I iElWjEWX,-  P,\\. 

/-i  —i 

For  the  case  of  a random  sum  of  random  variables  the  problem  of  minimizing  expected 
total  cost  is  written  as 

P3.  minimize  £[/(Aflf  ....  Xn)\  - £|  £ Vlk\X,~Xk | 

W 

n m 1 

+ I I I \Xi-pA 

J- 1 /-I  A-l  1 

- I EW^WX,-  Xk\ 

!</<*<» 

n m •** 

+ I If  I \x,-p,A  . 

i- 1 /-I  [a  — I 


A comparison  of  P2  and  P3  indicates  that  the  two  differ  only  in  the  expected  value  of  the 
second  set  of  summations.  However,  since  the  value  of  the  random  sum  of  independent  and 
identically  distributed  random  variables  is  given  by  the  product  of  their  expected  values,  then 
solving  P2  is  equivalent  to  solving  P3. 


'As  a noutioml  convenience  no  distinction  is  made  in  the  random  variable  P,  - ( a,.b ,)  and  the  value  of  the  random 
variable. 
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Since  P2  and  P3  are  equivalent  formulations  in  expected  value,  it  suffices  to  treat  only 
one  of  the  cases  in  detail.  In  the  plane,  the  Euclidean  distance  between  the  points  X,  and  Xk 
and  the  points  X , and  P,  can  be  represented  by 

\X,  - = [(Xj,  - x*,)2  + (x,2-  x*2)2]i/2 

and 

I Xi  - P\  - [(xyl  - a,)2  + (x,2  - A,)2]172. 

Hence,  P2  can  be  written  as 

P4.  minimize  E[f(Xx,  ....  Xn)\  - £ E[  Vjk][(xjX  - xtl)2  + (xj2  ~ x^)2]172 

+ £ - a,)2+  (x;2-  6)2]172, 

r-i  <-i 

where  it  is  assumed  that  all  random  variables  P,  are  normally  distributed  in  2 m dimensions.  Let 
P2m  — (a  | ft  |,  a2,  b2 am,  bm ) be  normally  distributed  in  2m  dimensions  with  mean  vec- 

tor A2m  and  positive  definite  covariance  matrix  V 2m.  Hence  the  frequency  function  of  P2m  is 
expressed  as 

fiJPiJ  - 7,  T^-rru=f  *~U1(pin,  - A2J  V2-J,(P2m  - Alm). 

Kin)  VI  ' 2m\ 

Here  it  is  assumed  that  the  random  variables  are  dependent,  i.e.,  the  correlation 
coefficients  are  greater  than  zero  for  some  of  the  random  variables.  Since  the  covariance  matrix 
is  positive  definite,  then  there  exists  a nonsingular  2mx2m  matrix  Q2m  such  that  if 

P2 „ = Q2m'  P2m,  the  P2m  - {a,,  bx.  a2,  b2 am.  bm\  are  independent.  See  Ref.  [22]  for 

a proof.  In  this  case,  the  new  mean  vector  is  QfJ  A 2m  and  the  variances  will  equal  the 
corresponding  eigenvalues. 

In  location  problems  it  could  be  assumed,  without  loss  of  generality,  that  the  random  vari- 
ables P,  are  independent  for  all  i,  i.e.,  the  locations  of  all  existing  facilities  are  considered 
independent.  In  this  case,  for  the  same  / the  two  random  variables  a„  b,  are  dependent  and 
their  frequency  distribution  is  a bivariate  normal  distribution.  The  above  transformation  Q2~ ' 
will  rotate  the  x and  y axes  through  the  acute  angle  <t>,  where 

tan  24>  - 2o-„».|(tr2  - a}), 

and  will  transform  the  density  function  to  a bivariate  normal  distribution  with  new  independent 
variances  equal  to  the  corresponding  eigenvalues  of  the  covariance  matrix.  If  <l>  — (ir|4),  then 
cra  - <rb  - <r,. 

Thus  in  this  paper  only  the  independent  case  is  discussed  since  all  other  situations  can  be 
reduced  to  it.  Let 

a, ~N(na,  <rj),  for  all /.  / - 1 m,  and 

b, ~N(nb,  (t b) , for  all  /',  / - 1 m. 

To  simplify  computations,  it  is  assumed  that  <r„  — <rb  " er,.  No  distributional  assumptions 
are  required  for  Vlk  and  Wh  since  only  their  expected  values  are  required  in  P4;  we  let 
E[  V ,k]  - m m and  £l  W if ] - /Z„. 
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In  the  Appendix,  the  expected  Euclidean  distance  between  the  points  X , and  P,  is 
obtained.  Substituting  the  expected  Euclidean  distance  in  P4  yields 


minimize  £ M, *[(*/.  - **,)2  + (x/2  - xk2)2]m 

*1  Kj<Kn 


/'-I  /-I 


where  H is  the  confluent  hypergeometric  function  defined  by  (A.  10)  (see  Appendix)  and  X 2 is 
defined  as 


0)  ’ . X2  ” (x,\  - m„)2  + (*/2  - Ma,)2- 

It  is  easily  established  that  the  objective  function  in  P4  is  strictly  convex  [2J.  Also,  based 
on  the  differentiability  conditions  at  the  optimum,  the  necessary  conditions  can  be  obtained. 
Taking  the  partial  derivatives  of  E[f(Xh  ...  , Xn))  with  respect  to  all  X,  and  setting  them 
equal  to  zero  gives 


(2) 

and 

(3) 

where 

and 


dElf]  Q A fi»(xji-xkl)  i /7 
dxj,  " “ D 


*-t  " ik 

k*j 

j - 1 n. 


2 V 2 O-,  H 2 2'  2*1 


bE[f]  n A Pjk(Xj2-Xk2)  .1  /Ta_ 

°',5  ^ |« 


1 7 

2 ’ 2o- 2 


k*j 

7-1 n. 


Pik-  k >j 

M/*“  m*/.  *<; 


W - [(*„  - x*,)2  + (x/2  - **2)  *1 l/2,  for  all  j,  k. 

Unfortunately,  if  any  two  new  facilities  j and  k have  the  same  location  at  any  time,  then 
^ik  — 0 and  the  partial  derivatives  in  (2)  and  (3)  are  undefined.  Eyster  et  al.  [8]  employed  a 
hyperboloid  approximation  procedure  (HAP)  to  eliminate  this  situation.  To  adopt  their 
approach,  a positive  constant  < is  introduced  under  the  square  root  in  Dlk\  consequently,  the 
partial  derivatives  always  exist.  Let  X,*  denote  the  modified  Dlk,  i.e., 

(5)  Xrt  - [(xn  - xk2)2  + (x/2  - x*2)2  + c] l/2. 

When  we  substitute  (5)  in  (2)  and  (3)  and  set  the  derivatives  to  zero,  the  following  iterative 
expressions  result: 


x,y+i>  - 


k*j 


(6) 
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*,y+i>  - 


* . 1 /ZfEjLu  „(l  2 

J 

a,*  l nr  ^ a„  i ? x3(*>  l 

5r“+TVy ,5»,"|2-j-  2<r? | 


■ • • I 

• . 


4.  CONSTRAINED  PROBABILISTIC  FORMULATIONS 

4.1  Chance  Constrained  Multifacility  Weber  Problem:  Case  I 

The  first  chance-constrained  formulation  of  the  multifacility  Weber  problem  considered  is 
that  involving  products  of  random  variables.  The  optimization  problem  is  given  as 

V”  n m _ J X 2 I 

T Z - T-  1 • - T~7 

!*/<*%«  L /-l  <-l  l i(T'  I 

(8)  subject  to  Djk  < ? for  all  7,  k, 

. 7-1 «• 

(9)  and  Pr(  M',/?,,  < f „)  > y/„  [ m 


where 


D/k  - [(xy1-x*,)2+(x/2-x*2)2),/2,  ^/-  “ [(x/1-a,)J+(x/2-b/)Jl,/J, 


and  £/k,  and  y/(  are  known  constants.  The  first  set  of  constraints  (8)  is  deterministic  and 
forms  a convex  set.  Thus,  the  only  probabilistic  element  in  P5  is  due  to  the  chance  constraints 
(9).  To  develop  their  deterministic  equivalent  representations,  the  following  approximation  is 
employed:  Let  the  random  variable  W be  normally  distributed  with  mean  /i  w *nd  variance  a Jv- 
Let  the  random  variable  R 2 be  defined  by  R2  — (xj  — a)2  + (x2  — b)2,  where  a N(j, i0,  a2) 
and  b ~ N(fih,  a2).  The  probability-density  function  of  R2  can  be  shown  to  be  [17] 

. — '—(y+k1)  [v  r.,1 


SfW  ^ '°  ^ ’ ° < 9 K 


where 


X2  - (X]  - n„)2  + (x2  - m»)2 


1„  - the  modified  Bessel  function  of  the  first  kind  and  of  order  n. 

If  we  assume  that  W and  R2  are  independent,  then  the  probability  density  function  of  the  new 
random  variable  Z — WR  is  given  by 


g(z)  - 


I 

K<vr,j>/2(z) 


-1—^-2  Vi  V2 

2 p — L r — i 
2 2 


, 0 < Z < oo, 


. ' . - ' . ■ " 

- . - - 
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where  Kv  (ax)  denotes  the  modified  Bessel  function  of  the  second  kind  and  order  v,  and  v,  and 
v2  denote  the  degrees  of  freedom  for  noncentral  chi-square  distributed  random  variables. 

To  motivate  the  approximate  deterministic  equivalent,  let  Y - W2R 2,  where  R 2 is  distri- 
buted as  a noncentral  x2(X2)  with  two  degrees  of  freedom  and  noncentrality  parameter  A2,  and 
H''2  is  a noncentral  x2(m2)  with  one  degree  of  freedom  and  noncentrality  parameter  n2  (17).  If 
we  use  Patnaik’s  noncentral  chi-square  approximation  [23],  two  different  \2  distributions  with 
degrees  of  freedom  v,  and  v2,  respectively,  are  obtained. 

The  Mellin  transform  of  x2  is  given  by  Webb  [30]  as* 


(!2)  A/(/(x2,  | s)  - 2"-1’ 


Since  the  Mellin  transform  of  the  product  of  random  variables  is  given  by  the  product  of  their 
Mellin  transforms,  then 

(13)  M(g(y) | s)  = A/(/(x2,)|  s)  ■ s) 


+ s — 1 


- 2 


2<v-l) 


r 

vt 

y + i~i 

r 

v2 

f +s-l 

r 

pi  1 
2 1 

• r 

P2 

2 

In  order  to  find  g(y),  the  density  function  of  y,  the  inverse  Mellin  transform  of  (13) 
must  be  obtained: 


(14)  M~'  - g(y) 


1 


Vi  Vj 

r — r — 

2 2 


2 iri 


vi 

-r  + s-l 

v 2 

T -r  + x-1 

2 

2 

ds. 


Equation  14  can  be  expressed  alternatively,  in  order  that  the  inverse  will  be  recognized  easily, 

V t Vi 

from  the  tables  of  inverse  Mellin  transform.  In  (14),  let  s’  - 2s  + — + — - 2;  then 

j 2 2’ 

ds  - — ds  and  (14)  can  be  written  as 


(15) 


g(y)  - 


r|Tlrlf 


1 /»r  + /oo  • 

T~T  / (^)*>»"-* 

2iti 


r'T  + T-T"' 


T + f-T'"’' 


•The  Mellin  transform  of  the  random  variable  x is  defined  as  ft*’  ). 


540 


A.  A.  ALY  & J.  A.  WHITE 


In  Ref.  [7],  on  p.  331,  the  following  Mellin  transform  is  given: 
(16)  KAax ) - M~' 


r,f  + 2 


where  kv(ax)  is  the  modified  Bessel  function  of  the  second  kind  and  order  v.  Observing  the 
similarity  between  (15)  and  (16),  we  conclude  that  the  probability  density  function  for 
Y - W2R2  is 


(17) 


*00 


T + T-1 

» 4 4 


2 


To  simplify  (17),  let  y'12  - r,  then  2zdz  = dy.  Thus,  the  probability  density  function  for 
Z — WR  can  be  written 

Z K ( y,  -V  )/2  (Z) 

g(z)  “ — “ ~ < 2 < 00 


Vi 

r 

V2 

2 

2 

which  has  the  desired  form. 

If  the  cumulative  distribution  of  i ?(z)  is  required,  some  assumptions  are  made  first  to 
obtain  a closed  form  for  the  integral  of  (11).  It  is  first  assumed  that  all  weights  (IV)  are  such 

n w _ 

that  0 ^ < 1;  this  is  achieved  by  dividing  each  weight  by  £ w>  + L Thus’ 

/-I  /-I  !</<*<» 

the  weights  can  then  be  defined  as  the  fraction  of  the  total  weight.  Patnaik  [23)  provided  the 
following  approximation  to  a noncentral  \ 2 with  degrees  of  freedom  v and  noncentrality  param- 
eter X: 

x'v^X2)  - c\ } 


(18) 

where 


c_  JL+li  and  /-  v+ 


v + X ' v + 2X 

Therefore,  when  the  distribution  of  IV,2,  which  is  x'i iV2)  distributed,  is  approximated  by  a cen- 
tral x }>  the  degrees  of  freedom  are 

(19)  + 

From  the  assumption  that  the  random  variable  W,  takes  values  below  one,  the  second  term  in 

(19)  is  always  a fraction. 

Given  two  chi-square  distributions,  each  with  degrees  of  freedoms  n\,n2,  respectively,  if 
n,  > n 2,  then 

(20)  Pr  <X2,  < f2)  < Pr  (Xn2  < 


MULTIFACILITY  WEBER  PROBLEM  541 

It  may  be  seen  that  using  nt  instead  of  n2  will  underestimate  the  probability.  Hence,  there  is 
no  overestimation  in  the  probability  in  (9)  if  we  assume  that  f defined  in  (19)  equals  two,  since 
the  difference  in  this  range  is  small  for  the  \ 2 distribution.  Since 

(21)  Pr  ( Wj'Rj,  (j,)  - Pr  ( W}  Rjf  < {}), 

applying  Patnaik’s  approximation  (x'JU2)  - c\j]  yields 

Pr  ( Wj  Rjt  < fj)  = Pr(  W},  Rjf  < fj/c.c^j), 
where  c,  and  c2  are  defined  as  in  (18). 

Under  the  assumption  that  v2  - 2,  (11)  reduces  to 

, 2 

(22)  - —r — j K(v  2)/2(z) , 0 < z < oo. 


v-'  v- 


The  cumulative  distribution  is  obtained  by  integrating  (22)  over  z.  Letting 

(23)  F(a j)  - Pr  (W„  Rh  ^ a/(). 
where  a},  - ^]\cxc-p]n  and  noting  that 

(24)  J/z-  Kn(z)dz  - -w("+l)  tf„+1( w)  + 2"  r(n  + l), 
substitution  of  (24)  in  (25)  yields 


(««)  2 Kv.  («/,) 


Fi«u)  ~ 1 - 


2 2 ' r y 


Therefore,  the  chance  constraints  in  P5  can  be  written  as 


a/i2  *.,„(«*>  forally'-l, 

I*.  I / T'  and  / - 1. 


1 “ Vi,  > 


Note  that  ah  is  a function  of  C|,  and  from  (18)  it  is  clear  that  it  is  a function  of  Xt\  in  the  same 
manner  vt  is  a function  of  Xt.  Since  Af„()  is  well  tabulated  and  available  for  computer  calcula- 
tions, an  iterative  method  to  solve  the  nonlinear  programming  problem  is  recommended.  Prob- 
lem P5  may  be  stated  in  a deterministic  equivalent  form  as 

P6.  minimize  Z - £ nJkDJk  + £ £?/,<>■,  «l-  y,  1,  - 

* i l </<*<«  v * /-i  ,-i  l 1 2a-;  I 

subject  to  filkDlk  < (lk,  for  ally,  k, 


For  a constant  v,,  it  is  easily  shown  that  the  last  set  of  constraints  forms  a convex  set 
(from  the  definition  of  *(•));  the  first  set  of  constraints  also  forms  a convex  set.  Thus,  the 
joint  constraint  set  is  convex.  It  can  be  shown  that  the  objective  function  is  strictly  convex; 
thus,  if  a local  optimum  is  achieved,  it  is  also  a global  optimum.  Hence,  a number  of  nonlinear 
programming  algorithms  may  be  employed  to  obtain  the  solution  to  P6.  In  the  case  that  v,  is 
not  considered  as  a parameter,  i.e.,  Vi  is  a function  of  XJt  the  convexity  condition  of  the  con- 
straints may  not  hold.  However,  a local  optimum  solution  is  still  available  and  it  may  turn  out 
to  be  a global  optimum  solution. 

It  is  easier  to  work  with  the  constraints  when  v,  is  treated  as  a parameter  since  K„|/2(-) 
will  have  the  same  order  during  all  iterations.  However,  this  may  be  accomplished,  without 
loss  of  generality,  if  the  known  (/ ia , nb)  are  rescaled  so  that  any  coordinate  takes  a value 
between  zero  and  one.  This  will  imply  that  X;,  is  bounded  as  0 < X„  ^ 1,  from  (8),  V)  ~ 2 
and  the  constraints  given  by  (26)  are  written  as 

a a K\(a,)  1 - y /,.  for  all  ij. 

which  will  simplify  the  computations  dramatically.  Note  that  the  optimum  solution  has  to  be 
adjusted  to  its  former  scale  so  that  the  total  cost  obtained  is  in  the  correct  units. 

4.2  Chance-Constrained  Multifacility 
Weber  Problem:  Case  II 

The  second  type  of  problem  to  be  considered  involves  a random  sum  of  random  variables. 
The  optimization  problem  is  formulated  as 

n m J X J,  I 

P7.  minimize  Z - £ M /**>/*  + -V/T  £ H - 1.  - — y 

xl  !</<*<«  V L /-I  i-l  Z<T'  > 

subject  to  n.  jkDjk  < for  all  j.k 

, , ) 7-1 n. 

and  Pr  £ > y/(.  , _ t m 
> 

where  Djk  , R„,  y,»  H,  and  X/(  are  as  defined  previously.  In  order  to  solve  P7  the  chance 

constraints  are  converted  to  equivalent  deterministic  constraints.  If  we  define  the  random  vari- 
able Y , , as 

» ** 

yh  - t R}h\ 

h-\ 


we  may  conclude,  under  very  general  conditions,  that  Y h is  approximately  normally  distributed 
with  mean 
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and  variance 
(28) 

From  (A. 12), 
(29) 


n Y„\  = E[W,]V[R,\  + nw^EHR,} 

- VF" «(- f ^ 


and 

(30)  ElRfi-  la}  +Aj, 

For  a detailed  discussion  of  the  conditions  underlying  the  Central  Limit  Theorem  for  the  sum 
of  a random  number  of  independent  random  variables,  see  Blum,  ei  al  [31  and  Rdnyi  [24],  [25], 

The  chance  constraints  can  be  written  in  normalized  form  as 


(31)  Pr 


y„  - El  Yj]  s th-E[Y„  1 


SD[K„1 


SD(K„1 


> y i,  for  all  j,i. 


where  SD[Y,)  denotes  the  standard  deviation  of  the  random  variable  Yjr  Equivalently,  (31) 
can  be  expressed  as 


(32) 


f*  > Eir„]  + SDIYJ*-'  iyj. 


1 n, 

■ m. 


The  deterministic  equivalent  of  P7  is  given  by 


P8.  minimize  z - £ nikD,k  + ~ J 'f  £ in„<r,  H-  1,  - ^ 

*i  K/<K/I  w L /-I  ,-l  1 2cr, 


subject  to  ^ £/*.  1 ^ j < k ^ n, 


and  El  YJt]  + SDl  Yj,H~Hyj,)  < ‘ ’ m 


It  can  be  shown  that  E[Yj]  is  a convex  function  and  the  objective  function  is  strictly  con- 
vex [2).  However,  the  convexity  of  SDlY^]  may  not  hold.  Hence,  a local  optimum  solution  is 
guaranteed  using  any  convergent  convex  programming  algorithm. 


APPENDIX 


1.  Derivation  of  £l(x,-a)2  + (jc2— fc)2}: 
Let  R2  - (x{-a)2  + (x2-A)2.  From  (10) 


ffojO’) 


1 

2<r2 


2a1 


(j’+A1) 


i°  o < y < 00  • 


(A.l) 
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The  expected  value  of  R2  is  derived  as  follows: 
ElR*\  - fQ  y gRi(y)dy 


-X 


7<t2 


0 2( T2 


A Jy 


dy 


-2  A2 


1 


Let  A - - — ^ - w,  then  dw  — ~r~r  dy 

r *• 


(A. 2) 


2 <r 2 ’ 2o-2  2 o- 2 

E[R2]  “ 2o-2  e~*7  we~K  I0(2ky/w  ) dw 


But  from  Ref.  [1],  p.  375, 

(A. 3) 


/„(*) 


I 


*-o 


(A!)2 


Substituting  (A. 3)  in  (A. 2)  yields 

— t 00  2A  /<  oo 

flR2]  — 2cr7  e~K  X (^i)2~  X (yfw)2k  dw. 


It  can  be  seen  that  the  integral  is  a gamma  function  where 

T(z)  - f“  r1'1  e~'dt 
Jo 

from  Ref.  [1],  p.  255.  Therefore,  the  integral  equals 

(A.4)  f°°  w*+1  e~wdw  - r(A+2)  - (A  + 1)(A!) 

Jo 

Substituting  (A.4)  for  the  integral,  we  obtain 
EIR2)  -2<r2e-*'1  £ • (A+l) 

k-0  k- 


— 2o-2  e ** 


y (A2)*  y A(  A2)* 

A!  ** 


k — 0 


“ r t!  * 
Since  e2-  £ 

*-o  * ■ 


EIR2]  — 2o-2  e-*2  |e^+  (A2)  e* 


r* 


A! 


— 2o-2  + 2o-2  A2. 

Since  A2  - —r, 

2a2 

(A. 5)  E[R*\  -2o2  + A2. 

2.  Derivation  of  )2+  O-A,)2]1'2]: 
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Given  that  the  random  variable  P behaves  according  to  the  normal  distribution,  i.e., 
a — N(fia.(r2),  b ~ N(nh,a2),  we  let  R denote  the  statistic  given  by  R - [(jc, -a)2  + 
(x2-/>)2] l/2.  Then  the  probability  density  function  of  R is 


where 


e 2<r2 
<T2 


J— 


/„  — , 0 < y < oo, 

CTl 


X2  - (x, -fia)2  + (x2-m»)2.  and 

In  - the  modified  Bessel  function  of  the  first  kind  and  order  n. 


The  probability  density  function  of  R2  is  given  in  (15).  Since  the  statistic  R equals  the  square 
root  of  the  statistic  R2,  then 

£*(')  - gR(Jy)  ■ ^ . 

where  r - Vy;  thus  2 rdr  - rfyand  the  Jacobian  - 2 r.  Therefore, 

dr 

i — ^(/-2+^2)  , ) 

” 2r  2^  1 ^ l°  It2  ’ 0 < y K °°' 

which  gives  the  probability  density  function  shown  in  (A. 6)  Given  the  probability  density  func- 
tion of  R,  we  derive  the  expected  value  as 


£[*1  - f0  r gR(r ) dr 

r°°  r 2 — 

- f e 2(r 

J 0 


, Xr 
>o  J dr 

<T 


Using  the  expansion  of  l„(z)  given  in  (A. 3), 


rr  2 y ' g 2*>dr 

<T2  J0  kt  (k\)2 


rl  (t 

Let  - — r - w,  then  r — >/5crVw,  dr  — —=  w~l,2dw,  giving 
la1  V2 


£•(/?]  - 


, 2<r2  - 1 2<rJ  I 

*?n  (*!)2 


J*o~  (vf  o-)2*+2h-*+1  e~w-Zj=w  2 dw 


_ “~T  ” I -\/2cr  I r« 

” ^ e X J0  » ‘ 


t 
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But  the  integral  is  a gamma  function  [1],  p.  255,  thus 


(A. 8) 


*■  dw  = r 


*4 


Substituting  the  value  of  the  integral  in  (A. 7),  we  obtain 


9)  £■(/?]  _ J2 


K2 

k 

, 3 

a2 

r 

k +~ 

— T 00 

2<r2  r _ 

2it2 

2 

2-r2  y * u 

A fc!  r(*+D 


r(i) 


rd) 


1 


From  Ref.  [1],  p.  504, 
(A. 10) 


H(a,b,z ) 


T(a+*)  r(b) 


k-0 


r(a)  r (b+k)  k\  ' 


1 

1 

e 21,2  H 

l 1. 

X2 

2 

2 

2<r2 

Substituting  (A. 10)  in  (A. 9)  we  obtain 

E[R)  - J2a  T 

—I  — ! ~ 

2 j " V 2 ’ 


Since  f 


(A.l  1) 


Equation  (A.l  1 ) may  be  further  simplified  by  using  the  Rummer  Transformation  (Ref. 
Cl  I,  p.  505)  which  is  stated  as 

H(a,b,z)  — e:  H(b—a.b,—z) 

Application  of  the  Rummer  Transformation  to  (A.  11)  yields  the  following  expected  value  of  R , 


(A. 12) 


E[R\ 


2a1 
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ABSTRACT 


The  shortest  path  problem  between  two  specified  nodes  in  a general  net- 
work possesses  the  unimodularity  property  and,  therefore,  can  be  solved  by 
efficient  labelling  algorithms.  However,  the  introduction  of  an  additional  linear 
constraint  would,  in  general,  destroy  this  property  and  the  existing  algorithms 
are  not  applicable  in  this  case.  This  paper  presents  a parametric  approach  for 
solving  this  problem.  The  algorithm  presented  would  require,  on  the  average, 
a number  of  iterations  which  is  polynomial^  bounded  The  similarity  of  this 
approach  to  that  of  the  generalized  Lagrange  multiplier  technique  is  demon- 
strated and  a numerical  example  is  presented. 


1.  INTRODUCTION 

The  problem  of  determining  the  shortest  chain  between  a pair  of  specified  nodes  in  a gen- 
eral network  is  of  interest  in  many  ways  [5],  The  problem  is  formulated  as  a special  case  of  the 
minimal-cost-flow  problem  [4],  and  efficient  algorithms  are  available  for  computing  the  shortest 
path.  The  shortest  path  so  obtained  will  minimize  a particular  linear  attribute  (function)  of  the 
path  such  as  cost,  time,  or  distance.  In  the  constrained  shortest  path  problem,  the  optimal  path 
must  adhere  to  an  additional  linear  constraint  and  minimize  the  chosen  attribute.  A typical  for- 
mulation of  this  problem  will  be  one  of  finding  a path  with  minimum  distance  subject  to  a 
budgetary  constraint.  Alternatively,  it  may  be  one  of  minimizing  the  cost  subject  to  a con- 
straint on  time.  The  introduction  of  such  an  additional  constraint  destroys  the  unimodulerity  of 
the  constraint  matrix,  and  any  simplex-based  algorithm  would  not,  in  general,  guarantee  an 
integral  solution. 

In  this  paper,  we  treat  the  additional  constraint  also  as  an  objective  and  formulate  a bicri- 
teria linear  program.  The  optimal  solution  to  the  original  problem  is  shown  to  be  a special  kind 
of  extreme  point  of  the  bicriteria  problem.  An  algorithm  is  presented  for  obtaining  such  an 
extreme  point.  At  each  iteration  of  the  algorithm,  a shortest  path  that  minimizes  a positively 
weighted  average  of  the  two  objectives  is  determined.  It  is  shown  that  one  needs  to  solve,  on 
the  average,  at  most  n such  problems,  where  n is  the  number  of  variables,  to  obtain  the  desired 
nondominated  extreme  point.  It  is  of  interest  to  see  that  the  process  has  strong  similarity  to 
the  generalized  Lagrange  multiplier  technique  [3],  and  this  aspect  is  demonstrated.  A numeri- 
cal example  illustrates  the  algorithm. 

2.  FORMATION  OF  THE  PROBLEM 
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(PI) 


Minimize 

*.</>  - I 

A 

a (x,  y) 

■fix. 

subject  to 

I fix.  y) 

- T fiy.  x) 

v € A/ 

and 

I bix,y)  ■ 

A 

fix.  y) 

< B, 

fix,  y)  = C 

1 or  1. 

1 if  x — s, 
-1  if  x = t. 

0 otherwise, 


It  is  assumed  that  aix,  y ) and  bix,  y)  are  nonnegative  for  all  (x,  y)  € A,  and  the  vector  b 
(with  bix,  y)  as  its  components)  is  not  a multiple  of  a. 


Consider,  now,  the  following  bicriteria  linear  program. 
Minimize  z\if)  = I a(x,  y)  ■ fix,  y) 
and  z2(/)  = I bix,  y)  • fix,  y) 


subject  to  £ fix,  y)  - £ fiy,  x) 

y€V  y€N 


1 if  x - s, 
-1  if  x = t, 

0 otherwise, 


(P2) 


fix,  y)  > 0. 


In  the  above  formulation,  it  would  be  rare  that  the  two  objectives  are  simultaneously 
minimized.  Therefore,  the  solution  of  the  problem  has  to  be  based  on  the  concept  of  nondom- 
inance [6].  Denote  the  feasible  set  of  (P2)  by  F,  and  let  zif)  denote  the  two  component  vec- 
tor [z \ if) , z2(/)].  Let  f*  6 F and  Gr  - / € F\  zif)  < zif*).  The  point  /*  is  said  to  be  a 
nondominated  solution  if  and  only  if  zif*)  — zif)  for  all  / 6 G f-.  Extreme  points  of  the  set  F 
which  are  nondominated  solutions  are  called  nondominated  or  efficient  extreme  points 

The  following  lemma  provides  the  relationship  between  (PI)  and  (P2). 

LEMMA  1:  There  exists  a nondominated  extreme  point  of  (P2)  which  is  a solution  to 

(PI). 


PROOF:  Let  /*  be  that  optimal  solution  to  (PI)  for  which  T bix,  y)  • fix,  y)  is  minim- 
ized. Then,  clearly,  f*  is  a nondominated  solution  to  (P2).  The  unimodularity  of  the  con- 
straint matrix  of  (P2)  implies  that  all  the  extreme  points  of  F are  integer  vectors.  Since  each 
one  of  these  extreme  points  of  Fis  a vector  of  zeros  and  ones,  it  follows  that  any  integer  solu- 
tion of  (P2)  must  be  an  extreme  point  of  F.  Again,  since  /*  is  an  integer  nondominated  solu- 
tion of  (P2),  it  must  be  a nondominated  extreme  point  of  (P2).  Thus  an  optimal  solution  to 
(PI)  can  be  obtained  by  searching  through  the  nondominated  extreme  points  of  the  decision 
space  F. 

Let  the  feasible  set  in  the  two-dimensional  objective  space  of  (P2)  be  denoted  by  Z.  Con- 
sider the  following  lemma: 

LEMMA  2:  The  set  Z is  convex  and  each  extreme  point  of  Z corresponds  to  at  least  one 
extreme  point  of  the  feasible  set  F. 
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PROOF:  The  convexity  of  the  set  Z is  obvious.  Consider,  now,  an  extreme  point  z ° of 
the  set  Z.  There  exists  at  least  one  /,  say  /°,  such  that  z(/°)  = z°.  Assume,  contrary  to  the 
hypothesis,  that  there  is  no  extreme  point  in  the  set  F which  corresponds  to  z Since  /°is  not 

k 

an  extreme  point,  we  can  write  /°=  £<*,/„  0 < a,  < 1,  where  the  /,’ s are  distinct  extreme 

* 

points.  So  z(f°)  - £ a ,z (/) . If  all  z(/,)’s  are  the  same,  then  each  must  equal  z °,  leading  to 

i-i 

a contradiction.  Otherwise,  z °,  an  extreme  point,  is  being  expressed  as  a convex  combination 
of  some  distinct  points,  which  again  leads  to  a contradiction. 

Every  extreme  point  of  F need  not  correspond  to  an  extreme  point  of  Z.  A point  z € Z 
is  nondominated  if  and  only  ifz  < z — z = z.  Thus  every  nondominated  point  in  the  decision 
space  F corresponds  to  a nondominated  point  in  the  objective  space. 

The  algorithm  presented  below  involves  a search  process  in  the  set  Z for  nondominated 
extreme  points.  Each  iteration  involves  the  solution  of  a shortest  chain  problem  in  which  the 
arc  distance  c(x,  y)  is  a positively  weighted  sum  of  a(x,  y)  and  b(x,  y).  Initially,  two  non- 
dominated extreme  points  are  determined  by  solving  two  shortest  chain  problems,  one  for  each 
objective.  The  positive  weights  are  determined  by  the  slope  of  the  line  joining  these  two 
points.  If  the  two  points  are  z(r)  and  z<5),  the  weights  would  be  w,  = zjr)  — z 2(,)  and 
*>2  — z i(,)  — z{r).  With  these  newly  defined  costs  on  arcs,  a shortest  chain  problem  is  solved. 
Thus,  either  a new  nondominated  point  is  revealed,  or  it  is  shown  that  the  weighted  objective 
function  does  not  change.  In  the  latter  case,  if /is  a solution,  then 

Y c(x,  y)f(x,  v)  = w,  • z{r>  + w2  • z2(s) 

(x.v)iA 

where  c(x,  y)  - w,  • z{r)  + tv2  ' z2°.  1°  this  case,  one  of  the  alternative  optima  is  the  desired 
solution  and  the  algorithm  terminates.  In  the  former  case,  the  new  nondominated  point 
obtained  is  used  to  determine  the  positive  weights  for  the  next  iteration. 

3.  ALGORITHM 

STEPO:  Find  z,(,)  - Min  (z,|/€F) 

and  z2<n  - Min  (z2| z i ” z/11  and  /€F). 

[z2(1)  is  obtained  by  searching  through  all  the  extreme  points 
that  yield  z/".] 

Record  (z,(l),  z2(1>).  Similarly,  find 

z2(J)  - Min  (z2|/€  F) 

and  Zi(2)  - Min  (z ijz2  ” z42)  and  /€F). 

Record  (z/2>,  z2<2))  and  set  k - 2.  Stop  if 

B < z2<2)  or  B > z2(1’,  since  in  the  first  case  the  problem  is  infeasible  and  in  the  second  case 
the  additional  constraint  is  redundant.  Otherwise,  set  r - 1 and  s - 2 and  go  to  Step  1. 

STEP  1:  Set  *}'■*>  - z{r)  - z2<0  and  w2(r’>  - z,(j)  - z}r). 

Let  / be  the  optimal  solution  to  the  shortest  chain  problem  with  c(x,  y)  - 
h ’{r,,a(x.  y)  + wj'0  ■ b(x,  y)  as  the  unit  cost  of  arc  (xt  y).  (If  there  are  alternative  optima, 
choose  the  one  for  which  Zj  is  minimum  and  call  it  /.)  If,  for  this  solution  /, 
I c(x,y)  • f(x.  y)  - w1<r-,,zI(,)  + w2(r,,z2(r),  go  to  Step  3. 

(x.y)i  A 


t 
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Otherwise  set  k - k + 1 and  go  to  Step  2. 

STEP  2:  Let  z,u>  - I a(x.y)  ■ /(x.  y) 

and  z2U)  - Z b(x,y)  ■ f(x,  y) 

<r.v>€  A 

If  z2u)  > B , set  r - k.  If  z2U)  < fl,  set  s - k.  If  z2u)  - B,  go  to  Step  4,  otherwise  go  to  Step 
1. 

STEP  3:  Determine  all  the  alternative  optima  for  the  shortest  chain  with  c(x,  >0,  as 
defined  in  Step  1,  as  the  unit  cost  on  arc  (x,  y)  and  choose  the  one,  say  /*,  for  which  the  addi- 
tional constraint  is  satisfied  and  Ea(x,  y)  • /*(x,  y)  is  minimized.  Stop,  since  /*  is  the  desired 
solution. 

STEP  4:  Stop,  since  7,  the  solution  which  yields  z(k>  and  z2U)  for  the  two  objectives  of 
(P2),  is  the  desired  solution. 

Certain  comments  can  be  made  about  the  algorithm.  Both  in  Step  1 and  Step  3,  the  algo- 
rithm would  have  to  determine  all  the  alternative  optima  to  the  shortest  chain  problem.  This  is 
not  difficult  to  do,  since  all  the  integer  solutions  satisfying  the  constraints  of  (P2)  are  extreme 
points  of  (P2),  and  there  are  simplex-based  algorithms  for  finding  the  shortest  chain  [1].  The 
simplex  method  can  be  used  to  obtain  the  shortest  chain  since  it  is  easy  to  show  the 
equivalence  between  the  shortest  chain  and  the  assignment  problem  [5].  Also,  Dijkstra’s  algo- 
rithm [2]  can  be  modified  to  yield  all  the  optimal  solutions  to  the  shortest  chain  problem. 

4.  VALIDITY  OF  THE  ALGORITHM 

Before  proving  the  validity  of  the  algorithm  let  us  introduce  some  notation.  Given  two 
efficient  extreme  points  z*"1,  z<v)  €R2,  let  Q(u,  v)  - z|z  < A • zUl)  + (1  - X)z<l),  Z|  > Z|("\ 
z2  > z2<l,).  Let  us  assume  that  the  algorithm  does  not  converge  in  Step  0,  so  that  problem  (PI) 
has  a nontrivial  feasible  solution. 

LEMMA  3:  Let  z be  a solution  in  the  objective  space  corresponding  to  an  optimal  solution 
of  (PI)  which  is  a nondominated  solution  of  (P2).  Then,  at  each  iteration  of  the  algorithm, 
z € Q(r,  s)  with  rand  sas  defined  in  the  algorithm. 

PROOF:  Let  us  first  show  that  the  algorithm  generates  an  efficient  extreme  point,  zU),  at 
each  iteration  of  the  algorithm.  Since  the  algorithm  does  not  terminated  at  Step  0 of  the  first 
iteration,  by  assumption,  z<n  and  z<2)  are  distinct  efficient  extreme  points.  Thus  w,  and  *v2  are 
strictly  positive,  and  hence  the  new  point,  if  any,  obtained  at  Step  1 of  the  first  iteration  is  the 
result  of  minimizing  a positively  weighted  average  of  the  two  objective  functions  of  (P2). 
Hence,  this  new  point  must  be  a nondominated  extreme  point.  Thus,  at  each  iteration  z<r)  and 
z(,)  are  efficient  extreme  points. 

Since,  at  Step  0,  t{n  minimizes  Ia(x,  y)  • /(x,  y)  and  z2(2),  minimizes  ZMx,  y)  • 
/(x,  y),  and  z(l)  and  z<2)  are  nondominated  extreme  points,  it  follows  that  initially  the  result  of 
Lemma  3 holds.  Now  assume  that  the  result  holds  till  iteration  k.  Then,  as  proved  earlier, 
z<*+»  is  a nondominated  extreme  point.  Thus,  the  interior  of  the  convex  hull  generated  by 
z(r),  zu+1),  and  z(,)  does  not  contain  any  efficient  point.  This  is  illustrated  geometrically  by 
Figure  1. 


Again,  since  rectangle  R cannot  contain  any  efficient  points,  Q(r,  k + 1)  U Q(k  + 1,  s) 
contains  the  point  z.  However,  depending  upon  the  line  z2  - fl,  either  Q(r,  k + 1)  or 
Q(k  + 1,  s)  contains  the  desired  point.  Hence,  z € Q(r,  s)  at  every  iteration  of  the  algo- 
rithm. 

The  algorithm  terminates  when  minimizing  the  objective  function  corresponding  to  the 
line  joining  z<r)  and  z(°  does  not  yield  any  new  point.  Thus  z|z  - {\z(,)  + (1  - A)z(s))  defines 
a supporting  hyperplane  in  the  objective  space  Z.  Also,  from  Lemma  3,  z € Q(r,  s)  at  termi- 
nation. Thus  z — A°z<r)  + (1  - X°)z(,>,  for  some  X°  between  0 and  1,  provides  an  optimal 
solution  to  (PI).  Step  3 of  the  algorithm  determines  a solution  which  corresponds  to  a point  on 
the  line  joining  z(r)  and  z(0  and  which  is  closest  to  z2  - B. 

Figure  1 illustrates  another  important  property  of  the  algorithm.  If  the  nondominated 
extreme  points  in  the  objective  space  are  uniformly  distributed  over  the  entire  nondominated 
region,  each  iteration  of  the  algorithm  divides  the  region  of  search  by  one  half.  Since  2"  is  the 
maximal  possible  number  of  feasible  solutions  to  (PI),  it  is  a crude  upper  bound  on  the 
number  of  nondominated  extreme  points  in  the  objective  space.  Hence,  the  maximum  number 
of  iterations  needed  to  reduce  the  search  to  one  efficient  point  would  be  log2(2")  (-«).  Based 
upon  Lemma  3 and  the  above  remarks,  we  can  state  the  following  theorem. 

THEOREM:  The  algorithm  determines  an  optimal  solution  to  (PI)  in  at  most  n iterations 
on  the  average. 

5.  RELATIONSHIP  TO  GENERALIZED  LAGRANGE  MULTIPLIER  TECHNIQUE 

Defining  z{(f)  - Ia(x,y)  • f(x,  y) 

(jr.y)  i A 

and  z2(/)  - I b(x,y)  ■ f(x,  Y), 

(r.p)€  A 

the  problem  (PI)  can  be  rewritten  as: 

Minimize  Z|(/) 

subject  to  z2(/)  < B, 

ftF. 
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Now  consider  the  problem  of  minimizing  w,  ■ zx(f)  + w2  • z2(/)  subject  to  / € F , where  tv, 
and  w2  are  determined  at  Step  1 of  the  algorithm.  If  we  define  A = this  is  equivalent  to 

minimizing  z//)  + Xz2(/).  The  weights  thus  provide  a Lagrange  multiplier  at  each  iteration  of 
the  algorithm.  The  nature  of  the  algorithm  is  therefore  similar  to  the  GLM  method  proposed 
by  Everett  [3],  It  is  important  to  note,  however,  that  the  algorithm  presented  here  is  a finite 
one  whereas  the  application  of  GLM  method  to  problem  (PI)  would  not  guarantee  finite  con- 
vergence. 

6.  AN  EXAMPLE 

Consider  the  following  undirected  network  shown  in  Figure  2.  The  first  number  on  an  arc 
(x,  y ) gives  a(x,  y),  and  the  second  b(x,  y).  The  additional  constraint  in  the  problem  is 
LMx,  y)f(x,  y). 


Figure  3 illustrates  the  application  of  this  algorithm  to  the  network  in  Figure  2.  The 
desired  solution  is  not  an  extreme  point  in  the  objective  space,  but  corresponds  to  an  extreme 
point,  that  is  a chain,  in  the  decision  space  F. 


T 
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7.  DISCUSSION 

Although  the  constrained  shortest  chain  problem  is  an  integer  problem,  the  algorithm 
presented  in  this  paper  would  solve  the  problem  efficiently  without  resorting  to  integer  pro- 
gramming techniques.  The  methodology  presented  is  quite  general.  An  additional  constraint 
can  be  handled  easily  in  any  0-1  integer  program.  The  algorithm  becomes  attractive  when  this 
integer  program  without  the  additional  constraint  has  a special  structure,  such  as  unimodularity, 
leading  to  a computationally  simple  solution  method.  Minimum  spanning  tree,  assignment,  and 
knapsack  problems  would  fall  into  this  category. 

The  method  can  be  easily  extended  to  the  situation  where  there  is  more  than  one  addi- 
tional constraint  by  "nesting."  Thus,  if  there  are  two  additional  constraints,  one  would  not  have 
to  solve  more  than  n 2 shortest  chain  problems,  on  the  average,  to  get  the  desired  solution. 
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ABSTRACT 

The  paper  provides  a new  theoretical  framework  for  generating  dominance 
conditions  and  lower  bounds  and  for  solving  special  cases.  All  existing  and 
new  results  have  been  derived  in  a routine  and  simple  manner. 


1.  INTRODUCTION 


Consider  the  following  problem:  Find  a permutation  P - px,p2, ■ ■ ■ ,P„  of  rows  1,2 n 

of  a given  n x m matrix  {/„},/„  > 0 that  minimizes 


(1) 


T(P,m) 


max 


w\  w2  m 

J-l  s-w,  J-H’  | 


over  a set  of  integers  w„_x  which  satisfy  conditions 

(2)  1 <**>!<  w2  < < wn_ | ^ m. 

We  have  just  formulated  the  well-known  flow-shop  problem  of  finding  a permutation  P 
that  minimizes  the  completion  time  T(P,m)  of  processing  items  1,2 . . . n on  machines 
M\,M2,  . . . ,M„  provided  that  the  processing  times  r„  of  item  r on  machine  Ms  are  given,  that 

each  item  passes  through  the  machines  in  the  same  order,  MVM2 Mm,  and  that  the 

machines  process  the  items  in  the  same  order. 

A sequence  y of  cells  of  an  n x m matrix  is  called  a segment  if  each  element  (r,s)  of  y 
(except  the  last)  is  followed  by  either  (r+l.s)  or  (r,s+l).  An  m+n-l  element  segment  is 
called  a path. 

REMARK  1:  Johnson  [5]  meant  by  a path  a walk  in  the  matrix  from  the  upper  left-hand 
corner  to  the  lower  right-hand  corner,  taking  steps  to  the  right  or  downward. 

Observe  that  the  summations  in  (1)  are  extended  over  the  following  path  T (T  makes  a 
downward  turn  at  each  (r,a>,),  r < n): 

r - (1, 1) .. . (1,W|);  (2.W,) ...  (2,h>2);  (/». *vw_i) ...  (n,m). 

T(,P,m)  — max  T t , 

r-frl  Prs 


r.in 


(M)«r 
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where  [T]  is  set  of  all  paths.  Path  T is  called  a critical  path  for  a given  permutation  Pif  T(P,m) 

“ £ btr 
r 

This  paper  offers  an  entirely  new  theoretical  framework  for  solving  the  problem  that 
makes  the  shape  of  the  segment  its  focal  point.  Then  the  derivation  of  all  existing  and  new 
results  in  such  areas  as  special  cases  111,  13],  15],  [8],  [9],  [11],  [12],  [14],  [15],  dominance 
conditions  [2],  [7],  [10],  [13],  and  lower  bounds  [4],  [6]  becomes  a simple  routine. 

We  have  shown  that  the  flow-shop  problem  can  be  formulated  in  pure  combinatorial 
terms  without  the  usual  Machine-Job-Completion-Time  interpretation.  Hence,  such  widely 
used  concepts  as  earliest  time,  shortest  run-out  time,  and  bottleneck  and  nonbottleneck 
machine  are  no  longer  necessary  to  develop  lower  bounds,  since  the  shape  of  the  critical  path  is 
the  main  factor.  The  author  hopes  that  further  extension  of  this  new  approach  will  ultimately 
produce  a reasonably  efficient  branch-and-bound-solution  method. 

2.  SELECTED  TYPES  OF  CRITICAL  PATHS 

We  find  it  convenient  to  describe  a segment  by  a sequence  of  R and  D symbols  that  indi- 
cate its  right-hand  and  downward  turns.  Notice  that  consecutive  symbols  of  this  sequence  are 
different.  Figure  I illustrates  a DRDR  segment.* 


Figure  1 


1 


(a)  (•')  (b)  (b'l  (d) 

Figure  2 

We  will  show  that  the  solution  of  the  flow-shop  problem  is  easily  available  whenever  the 
type  of  the  critical  path  T remains  the  same  for  all  permutations  P and  T is  of  the  following 
type:  (a)  RD,  (a  ) DR,  (b)  DRD,  (b  ) RDR,  (c)  RDRD,  (c’)  DRDR,  or  (d)  RDRDR  (see 
Figure  2). 

REMARK  2:  Each  case  includes  D and  R as  a subcase.  Cases  a and  a’,  b and  b\  c and  c' 
are  subcases  of  the  subsequent  cases  (but  not  of  each  other). 

n 

CASE  a:  Path  T passes  row  1 and  column  m where  £ trm  is  constant  for  each  P. 


m-1  m-1 

SOLUTION:  Find  min  £ („  — £f„.  Then  any  P - i — the  optimal  permutation. 

' i-i  <-i 

*r  makes  at  mosl  m-1  ft-turns  and  n-l  D-iurns 

te.g.,  P - in,  where  ir  is  an  arbitrary  n-1  element  permutation  of  numbers  r«  (1,2 n)  - (/). 
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CASE  a':  SOLUTION:  Find  min  — £ trs  — Then  any  P — ....  / * is  the  optimal 

' 5-2  5-2 

permutation. 

CASE  b:  It  is  well-known  that  Johnson’s  algorithm  solves  the  problem  whenever  T 
makes  exactly  one  R turn  (as  in  this  case). 

SOLUTION:  Apply  Johnson’s  method  to  the  two-machine  AB  flow-shop  problem  with 

nt  — l m 

processing  times  A,  =*  £/„  and  Br  - 

5-1  5-2 

REMARK  3:  The  solution  of  Case  b remains  optimal  under  a much  relaxed  assumption 
that  T makes  the  single  R turn  for  the  permutation  produced  by  Johnson’s  algorithm. 

CASE  b':  Path  T passes  through  an  entire  column,  say  column  h. 

SOLUTION:  Find 


h — \ m 

min  Z'o s + Z t„ , 

p*‘l  s-l  s-fc  + 1 


h — 1 m 

Z^'S  + £ t/S- 

5-1  5-/I  + 1 


Then  any  P = i ...  j\s  the  optimal  permutation. 

CASE  d:  Assume  that  T makes  two  D turns  along  columns  u and  v,  u < v. 


v — I 

SOLUTION:  Solve  (as  in  Case  b)  the  two-machine  AB  problem  where  A,  - and  Br 


V — // 

. df 


~ Z'm  and  let  1,2 n be  the  optimal  permutation.  Consider  sequences  pa  qilp,  1,  .. 

tt  + 1 

P~ + ) 9—1, <7+1 n,q  for  all  possible  1 < p ^ q < n.  Let  '( paq ) be  the  completion 

time  of  sequence  paq  on  machines  A and  B.  Find 


min 

!>*q 


u-l 


Z'/"  + Z 5 + tipaq) 

S-l  s— v+1 

Then  P — iaj  is  the  optimal  permutation. 


” Z'«  + Z I is  + t(iaj) . 

S-l  s-v+1 


CASES  c and  c’:  SOLUTION:  Proceed  as  in  Case  d. 


Hi 


Sequence  P — ip  - /,  1 /-I  ,/+l 


n is  optimal  if 


min 

!</><<■ 


Z'm  + tip  1 8) 

s-l 


- Z r„  + f(ifl). 

i-i 


while  sequence  P - <rj-  1,2 j-l.j+l n j,  is  optimal  if 


min 


Z tqs  + t(<rq) 

s-v  + 1 


- Z t/s  + t(<rj). 

S-V  + 1 


#c  g.,  P - w/  (sec  previous  footnote). 
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3.  CLASSIFICATION  OF  SPECIAL  CASES 

OF  THE  FLOW-SHOP  PROBLEM 

All  known  special  cases  belong  to  one  of  the  following  categories: 

CASE  I:  For  some  0 < h < m, 

min  r„  ^ max  f„+|,  1 ^ s < h-\, 

r r 

and 

min  <„+1  > max  /i+l  < s < m. 

r r 

Several  subcases  of  Case  1 have  been  solved  in  Refs.  [31,  (5),  [9],  [14],  and  [15]. 

CASE  II  [9]:  For  some  1 < h < m-l, 

min  f„+)  > max  1 < s ^ h— 1, 

r r 

and 

min  > max  rn+l,  h+l  < s < m-l. 

r r 

References  [1],  [3],  [8],  [14],  and  [15]  considered  special  cases  of  II. 

CASE  III:  For  some  i ^ n,  and  h < m, 

t,S  > I rs-  1 < r < ». 

and 

t rh  > tn,  1 < s < m. 

For  subcases  of  Case  III  see  Refs.  [11],  [12],  and  [15]. 

CASE  IV:  min(rH,/fm)  ^ f„,  1 < r ^ n,  2 < s < m-l.  Two  subcases  of  Case  IV  have 
been  solved  in  Ref.  [15]. 

The  envelope  concept  (see  the  following  section)  will  be  utilized  in  solving  these  special 

cases. 

4.  ENVELOPES 

Let  y be  an  arbitrary  segment  having  two  endpoints  (/,«)  and  O'.  v ),  / < j,  u < v.  Such 
a segment  consists  of  j-i  + v-u  + \ cells.  Both  RD  and  DR  segments  connecting  the  same  end- 
points as  y are  called  envelopes  of  y and  denoted  by  y and  y respectively.  Note  each  of  the  seg- 
ments y,  y , and  y passes  through  j— t + v— u-t-1  cells.  If  i—j,  or  u — v,  then  y — y — y*  Every 
path  T can  be  presented  as  a sequence  of  segments  ?|,  y7.  ■ ■ ■ •>*,  where  the  last  cell  of  y,  is  an 

initial  cell  of  y,+1,  t — 1 k- 1.  Otherwise  the  y,  are  mutually  exclusive.  The  envelope 

concept  makes  it  easier  to  visualize  the  shape  of  T whenever  it  is  presented  as  a sequence  of 
segments.  For  instance,  T — ^t  >2  indicates  that  T is  a DRD  type  (see  Figure  3 where  y\  and 

y2  share  cell  (r.s)),  while  T - T shows  that  T is  a RD  path. 


•Then  both  envelopes  are  R or  D segments,  or  single  points  (if  i-/'.  and  u-v). 


Figure  3 


REMARK  4:  Paths  and  segments  will  be  denoted  by  symbols  T and  y exclusively. 

The  solution  of  Cases  I-1V  requires  a preliminary  proof  that,  for  every  segment  located  in 
a certain  area,  one  of  the  following  conditions  holds: 


(3) 

I v < 

£ t^  for  each  P, 

(r.s)€y 

(r.s)«> 

or 

(4) 

M 

/A 

£ tp  s for  each  P. 

r y 


This  proof  always  follows  the  same  routine:  We  establish  a one  to  one  correspondence 
between  cells  ( r,s)ty  and(r',s')«y  (or  y,  depending  on  whether  we  want  to  prove  (3)  or  (4)), 
and  prove  that,  for  each  (r,s)«y, 

(5)  < V- 

We  say  that  the  diagonal  technique  or  the  row-column  technique  has  been  used  if  cells  (r,s)  and 
( r',s' ) are  located  on  the  same  45°  diagonal  ( r+s  - r'+s'),  or  on  the  same  row  (r-r  0 or 
column  (s—s 

5.  SOLUTION  OF  CASES  I AND  II 

CASE  I:  We  distinguish  three  subcases,  1 < h < m-1,  h — 0,  and  h — m. 

1.  Consider  the  first  subcase:  Let  T — ytfi,  where  y\  and  y 2 share  a cell  0,/i)«r  (see 
Figure  4).  For  /» —2,  m-1,  this  cell  is  specifically  defined  as:  {i,h)  - min(r,/i)«r  (if  h—  2),  and 

i 

( i.h ) - max(r,/?)«r  (if  h - m-1).*  Define  T0  - yi  Applying  the  diagonal  technique  to 

i — 

prove  (5),  we  can  see  that  (3)  and  (4)  hold  for  y - y2  and  y = -y,  respectively. 

Hence, 

£ Vs  ^ ^ tPr,‘ 

<r5>*  f (M)*r» 


mtm cm«  t,  (if  u »na  yj  (if  h-m-\)  make  a single  R turn  along  row  I. 
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Figure  4 


Consequently,  the  type  of  the  critical  path  is  DRD.  For  solution  see  Case  b. 

2.  If  h = 0,  then  min  f„+1  3*  max  tn,  1 < s < m -1. 

r r _ 

Again  applying  the  diagonal  technique,  we  see  that  (3)  holds  for  y = T.  Hence  I\  a RD  type, 
is  a critical  path.  For  solution  see  Case  a. 

3.  If  h = m,  then  min  irs  ^ maxr„+1,  1 < s < m—l. 

r 

Proceeding  as  in  the  previous  case  we  can  see  that  the  critical  path  is  a DR  type  (see  Case  a’). 

CASE  II:  Let  y = yty2y 3,  where  y,  and  y2  share  cell  (i.h),  while  y2  and  y3  share  cell 
(J,h  +1).  Consider  T0  = y\  y2  y2  (see  Figure  5).  Observe  that  y2  makes  one  R turn  since  it 
occupies  two  columns  h and  /i+l  only.  Utilizing  the  diagonal  technique,  we  can  see  that  (3) 
and  (4)  are  satisfied  for  y = y\  and  y = y2  respectively.  Therefore 

r ro 

which  means  that  RDRDR  is  the  type  of  the  critical  path.  For  the  solution,  see  Case  d,  where 
u — h,  and  v — h+l,  Ar  = trh  and  Br  = trh+ 1. 
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6.  DECOMPOSITION  OF  THE  FLOW-SHOP  PROBLEM 

CASE  III:  We  will  prove  the  following: 

PROPERTY  1:  Term  tjh  = max  tn  appears  in  (1)  for  every  P. 

Proof:  Let  P = 1,2 , n.  It  is  sufficient  to  show  that  cell  (i,h)  belongs  to  the  critical  path. 
Consider  a path  T that  does  not  pass  through  (i,h)  but  crosses  column  h at  (J,h)  and  row  / at 
(/,«).  Let  T = yiy2y)  where  y2  is  a segment  that  connects  end  points  (/',«)  and  (J,h).  Define 
r0  as  y \y2y2  or  yiy2y3  depending  on  whether  (i,h)  is  located  below  or  above  I\  Obviously 
(i,h)er0r  The  row-column  technique  leads  us  to  the  conclusion  that  (4)  (or  (3))  holds  for 
y = y2.  Thus 

Z In  < Z Q E D. 

r ro 

REMARK  5:  It  is  easy  to  show  (by  a counterexample)  that  Case  III  is  a necessary  condi- 
tion for  Property  1 . 

Property  1 allows  us  to  break  the  original  problem  into  2 subproblems.  This  can  be 
accomplished  in  the  following  manner:  (1)  Let  / =■  (1,2,  . . . , n),  and  generate  all  2"~l  subsets 
J of  /-(/).  (2)  Let  / denote  a complement  of  /,  and  consider  two  flow-shop  problems:  one  for 
jobset  /+(/)  and  machines  A/jAf 2,  ....  Mh,  the  other  for  jobset  /+(;)  and  machines 
Mh,  , Mm.  (3)  Find  sequences  of  the  type  ai  and  ip  that  minimize  the  sum  of  processing 
times  of  both  problems.  Then  permutation  P — aip  is  the  optimal  solution  of  the  entire  prob- 
lem. 


Consider  Case  III  for  h = 1 and  P = ...  /.  The  following  obvious  property  holds: 

PROPERTY  2:  Sequence  P maximizes  (1). 

PROOF:  Observe  that  Z <rrs  ~ Z^i  + Z*«  *s  largest  value  of  (1)  for  all  possible  P. 

r r /—i  s-2 

7.  ORDERED,  SEMIORDERED  AND  OTHER 

SPECIAL  CASES 

ORDERED  CASE  [11]:  (1)  For  each  / and  j,  row  / is  either  greater  than,  equal  to,  or 
smaller  than  row  j (i.e.,  for  all  1 < s < m:  either  ti5  ^ tjs,  tis—tjs,  or  t,s  < tjs).  (2)  For  any  u 
and  v,  column  u is  either  greater  than,  equal  to,  or  smaller  than  column  v (i.e.,  for  all  1 < r < 
m:  either  tru  > try,  im  - trv,  or  t„  < ij. 

SEMIORDERED  CASE  [15]:  The  first  condition  of  the  Ordered  Case  is  met.  It  is  obvi- 
ous that  the  Ordered  Case  is  a subcase  of  Case  III.  Let  h be  the  largest  column.  For  h - 1 the 
following  property  holds  [11]: 

PROPERTY  3:  (a)  No  critical  path  makes  a R turn  along  row  / if  some  subsequent  row  j 
(where  P is  greater  than  /. 

(b)  No  critical  path  makes  a D turn  along  column  u if  some  subsequent  column  v is 
greater  than  u.  Notice  that  Property  3b  does  not  depend  on  permutation  P. 
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Property  3 can  be  easily  shown  by  applying  the  envelope  approach  along  with  the  row- 
column  technique  from  Section  3.  According  to  Ref.  [11],  sequence  P is  optimal  if  all  n rows 
are  arranged  in  a nonincreasing  order.  Case  h = m is  symmetrical  to  case  h — 1.  Hence  the 
rows  in  the  optimal  permutation  are  arranged  in  a nondecreasing  order.  If  1 < h < m,  then 
the  problem  is  decomposable  as  Case  III  (see  Ref.  [12]). 

Let  h = 1.  Assume  also  that  m is  the  second  largest  column.  Then  Property  3 implies: 

COROLLARY  1:  The  critical  path  makes  a single  R turn. 

According  to  Ref.  [15]  (again  with  the  help  of  the  envelope  approach)  Corollary  1 is  valid 
for  the  following  less  restricted  cases:  (a)  Case  IV,  m = 3,  and  (b)  Case  IV,  m arbitrary,  pro- 
vided this  case  is  semiordered. 


Consider  the  following  two  cases: 

CASE  1:  trX  = r,2  = • • • = Lm  for  each  1 < r < n. 

CASE  2:  t is  = t2s  = . - . = t„s  for  each  1 < s ^ m. 

It  is  easy  to  see  that  the  critical  path  for  Case  1 makes,  for  each  P,  a single  R turn  along 

n m 

the  largest  row,  say  row  i.  Since  T(P,m)  = £frl  + £rjS  is  the  same  for  each  P,  any  sequence 

;-l  s-2 

is  optimal.  The  same  conclusion  is  valid  for  Case  2,  since  (1)  is  constant  for  each  permutation. 
8.  DOMINANCE  CONDITIONS 


Consider  sequence  p = pxp2.  ....  pk,  p C / = (1,2, ....  n).  Define 


(6) 


C(p,u,v ) 


max 


E(PlS  £ tp2s  + • • • + tpkS 


where  1 < w < v ^ m.  Then  (see  (1)) 


T(p,v)  = C(p,  l.v). 

Divide  p into  / disjoint  subsequences:  a,,  a2 a,  C /,  / < k,  preserving  the  order  of  the 

elements  of  p.  The  p - a(a2. . .a,.  From  (1)  and  (6)  we  obtain  the  following  formula: 

(7)  T(p,v)  - max  fr(a1(M,)  + CCaj.u^Ui)  + . . . + C(a,,u,_i,v)l 

The  following  three  dominance  conditions  assume  that  presequence  ir  C /of  permutation 
P — (tit  is  fixed. 

CONDITION  I [4],  [10]:  T(cr' ,u)  ^ T(<r,u ),  V 1 < u < m,  removes  sequences  cr  . . . 
(sequences  cr’  and  cr  contain  the  same  elements). 

PROOF:  Let  it  be  an  arbitrary  sequence  and  ir  O cr  — <t>. 
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In  view  of  (7), 

T(<rir,m)  - max  \T(<r,u)  + C(ir,u,m)],  and 

1 ^ m 

T((r'ir.m)  « max  [T(<t',u ) + C(n,u,m)]- 

I 

Hence  I “=>  [T(<r'ir  ,m)  < T(,<rir,m)],  Q- 

CONDITION  II  131,  {7),  1131:  Uaab.u)  - Titrb.u)  < tav,  V where  a, be/, 

eliminates  sequences  <rb . . . 

PROOF:  Consider  two  arbitrary  disjoint  sequences  n'  and  it"  where 
it'  U it"  — /-  trab.  Again,  by  (7) 

fWrV'j)  = max  [T(.<rab,u)  + C(ir',«,v)  + C(rr  ",v,m)l,  and 
T (a bit' a it". m)  - max  [ T(.crb,u ) + C(n'.u,v)  + C(o,v,w)  + C(n",w,m)] 

> max  [ T(crb.u)  + C(ir'.u.v)  + C(a.v.v)  + C(n",v,m )J, 

1 < w ^ v < m 


where  C(o,v,v)  - tav 


C(a,v,w)  - £fa 


Hence 

(g)  T{(rabir'ir"  ,m)  4 Tin  bit' air"  ,m) 

holds  if  for  each  u,  and  v,w  < v:  Tiaab.u)  < T(ab.u)  + tav,  which  is  Condition  II. 

CONDITION  III  [131:  TUab.u)  - T(<rb,u)  < tam  , V l<u<m,  removes  sequences 
cb ..  .a. 


PROOF:  Assume  ir"  - <t>  in  the  preceding  proof.  Then  it  is  obvious  that  (8)  holds  for 
if"  — $ whenever  T(<rab,u)  ^ Tfrrb.u ) + Cia.m.m ) ~ T(<rb,u)  + tan,  V u ^ m,  Q.G.D. 

It  is  known  (see  Refs.  {71  and  [131)  that  whenever  Condition  II  is  not  met,  there  exists  a 
hypothetical  example  where  (8)  is  violated.  To  show  it,  assume 

T(trob,u ) > T(arb.u)  + tav  for  some  u < v,  v > 2. 

Define  it'  - 0>)  and  rr”  - {q).  If  p and  q satisfy  (8)  and  (9)  of  reference  [131,  that  is: 


f/n 


< W |.  1 < S < tf-1, 

^ max  [r(tr<7^,J-H)  — T{trab,s) , T(<rb,s+\)  — T(<rb,s)],u  ^ s ^ v— 1, 

> T(<rbpa,s—l)  — T(crbp,s—\),  1 ” v> 

> 0,  v + 1 < s<  m; 
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< min  (fas+i,/,s+1),  1 < s < v-1, 

tqs  =-  > max  [rCcraftp.s+l)  — T(cabp.s),  T(cbpa,s+l)  — Tic bpa,s)],v  < s < m — 1, 
> 0,  s=m, 

where  the  tps  satisfy  (8),  then: 

1.  Ticabpq.m)  = Ticab.u ) + Cipq.u.m), 

V m 

2.  Cipq.u.m)  = £f„s  + Zv 


3.  Ticbpaq.m)  — Ticb.u ) + Cipaq.u.m) , and 


4.  Cipaq.u.m)  - £/„s  + ^ + ZV 


Hence, 


Ticabpq.m ) — Ticbpaq.m)  = Ticab.u)  — Ticb.u)  — tav  >0, 


Q.E.D. 


One  can  generate  the  following  dominance  conditions  that  are  independent  of  presequence 
<r,  and  involve  elements  a.bel only: 

CONDITION  IV  [16]:  Ciab.u.v)  < Ciba.u.v),  V 1 < u < v < m,  eliminates  sequences 
...ba...  . 

PROOF:  For  any  c,n  C /,  c n n = <f>, 

Ticabn.m)  — max  [T(<r,u)  + Ciab.u.v)  + Cin.v.m)],  and 
1 

Ticban  ,m)  — max  [Tic.u)  + Ciba.u.v)  + C(ir,v,m)]. 

1 v^m 


Hence  Condition  IV  —=> [Ticabn.m)  < Tic-ban, m)]. 


Q.E.D. 


Condition  IV  can  be  verified  in  the  following  manner  ( see  Ref.  [16]):  Consider  a flow- 

shop  problem  where  machines  M\M2 Mm  pass  through  jobs  a and  b in  order  ab.  If 

M\.M2 Mm  is  the  optimal  sequence  obtained  by  Johnson’s  method,  then  Condition  IV 

holds,  and  we  eliminate  sequences  ...ab...  . If  ....  A/,  is  the  optimal 

sequence,  then  we  eliminate  sequences  . . . ba  . . . (Then  Condition  IV'  holds  — see  Remark 
6). 

CONDITION  V:  C(ab,k,u)  — Cib.k.u)  < lav,  eliminates 

sequences  ...b  ...a  .... 

PROOF:  Consider  dominance  condition  II.  Then 

Ticab.u)  — Ticb.u)  — max  [ Tic.k ) + Ciab.k.u)]  — max  [Tic.k)  + Cib.k.u,)] 

!<*<u  K*<u 

< max  [Ciab.k.u)  - Cib.k.u)]. 
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Hence  V —^-Tiaab.u)  — Ticrb.u)  < /av]. 

Condition  V can  be  rewritten  in  the  following  form: 

f w w — I 1 


max 

A < * ^ w 


2>os 

S-* 


s-k 


Q.E.D. 


which  means  that  the  total  waiting  time  of  item  b for  sequence  ab  processed  on  machines 
Mk,  , Mu  does  not  exceed  tav. 


REMARK  6:  According  to  Ref.  [13],  pp.  1253-1254,  one  may  arrange  the  machines  and 
the  items  (i.e. , the  rows  and  columns  of  matrix  [r,,])  in  a reversed  order  and  find  symmetrical 
dominance  conditions. 


Applying  this  approach  to  V we  get 


CONDITION  V': 


max 

k 


E'fe  - 


w+1 
I.' as 


S — V 


^ h >«• 


f 1 O ^ I O $ 


which  also  eliminates  sequences  ...b  ...  a 


Thus  sequences  . . . b . . .a  . . . are  removed  whenever  V or  V is  met. 

REMARK  7:  Sequences  b . . . can  be  eliminated  if  Conditions  II  (a  — <fi ) or  III'  hold. 

All  dominance  conditions  that  involve  a and  b only  (this  includes  II  and  III  for  <r=<t>)  may 
improve  the  efficiency  of  the  solution  process,  since  they  do  not  require  excessive  computa- 
tions, and  their  verification  may  significantly  reduce  the  number  of  branches. 

We  offer  the  following  practical  suggestions: 

• Check  the  dominance^ conditions  in  the  following  order:  (1)  IV,  IV1;  (2)  II,  IT  ( a = 
<t>)\  (3)  V,  V'.  Condition  V (V  ) should  not  be  checked  if  IV  (IV')  or  II  (II')  does  not  hold.* 

• The  choice  of  a "forward"  or  "backward"  procedure  (See  Remark  6)  should  depend  on 
whether  the  number  of  sequences  b...  removed  by  Condition  II  exceeds  the  number  of 
sequences  ...  a removed  by  Condition  II'. 

• Prior  to  the  branch-and-bound-solution  procedure,  find  an  initial  solution  P using 
Johnson’s  Method  (as  in  Case  b,  Section  2). 

Determine  T(P,m)  and  t(P)  the  completion  time  of  sequence  P of  the  two-machine  AB 
problem.  If 

T(P.m)  - t(P ) - £ "l  tn, 

r- 2 J-2 

then  Pis  an  optimal  solution  (see  Ref.  [15]). 


'Since  Condition  V is  stronger  than  either  IV  or  II  while  V'  is  stronger  than  either  IV'  or  II'. 
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9.  LOWER  BOUNDS 

Reference  [6]  generates  lower  bounds  based  on  the  bottleneck  machine  and  non- 
bottleneck machine  concept.  Consider  a permutation  P = <rn.  Without  loss  of  generality,  we 
may  assume  <x  — 1,2,...,  k,  0 ^ k ^ n—  2.  Let  [ir]  be  a set  of  all  ( n—k)\  permutations  of 
elements  k + 1,  k + 2 n.  In  view  of  (6)  and  (7), 

n<r7r.m)  - max  [r(cr.w)  + C(rr,w,m)]  > max  [r(o-.w)  + min  C(n.w.m)]. 

Define  [y  J as  a set  of  all  segments  of  the  same  type  that  connect  end  cells  (k  + 1,  w) 
and  ( n.m ).  Observe  that  ly  J is  a single  element  segment  if  y,  is  a RD,  DR,  or  D type  (ym  is 
always  a D type). 

Assume  that  the  optimal  sequence  for  the  respective  type  (see  Section  2)  is 
■nw  - k +1 , ....  n.  Then 

min  C(n.w,m ) ^ max  I tn=*Kw. 

rw‘Ir.1  (r.s)ty^ 

Hence,  the  lower  bound  LB(o-)  for  all  possible  permutations  cr  ...  is 
(n\  LB(o-)  - max  (r(o-.w)  + if  J. 

If  <t  - <f>  (fc  - 0)  then  (9)  provides  a lower  bound  for  T(P,m). 


EXAMPLES: 

1.  Let  yw  is  a RD  segment.  Then  (see  Case  a) 


“ It,,  + I + £ tm  • 

,-w  r-*+l  * + l s_w  r-k+\ 


2.  If  y sub  w is  a RDR  segment  then  (see  Case  b') 


h-\  n m 

Kw  - max  It,s  + I /r*  + I *]s 

s-w  ,_*  + ! s-A+1 


where 


/r-1  n 


l/»-l  n 


It,,  + £(,»+  I 0,  - min  Ir„  + I f*  + Id 

,-w  r-k+l  S-/I  + 1 (s-w  r-*  + l S-A  + I ) 

fc+1  < p,  q < n. 

3.  Assume  that  y,  is  a DRD  type.  According  to  Case  b,  the  optimal  solution  ir.of  the  AB 
problem  is  k+l,  ....  n. 


Kw  - max  I £ trw  + I tqs  + It,m  • 

* + (r-*+l  s-w  + l r-q  J 

Observe  that  Kw-t(irw)  - I tn  where  t(irj  is  the  (minimum)  processing  time  of 

s-*v+ 1 

sequence  irw  of  the  two  machine  AB  problem.  Consequently, 

LB(<r)  - max  [r(<7\H')  + t(nj  - I /„]. 
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Define  [yj  as  a set  of  all  segments  of  the  same  type  that  connect  end  cells  (k  + 1,  w) 
and  ( n,m ).  Observe  that  [y  J is  a single  element  segment  if  yw  is  a RD,  DR,  or  D type  (ym  is 
always  a D type). 

Assume  that  the  optimal  sequence  for  the  respective  type  (see  Section  2)  is 
irw  — /c+1,  ....  n.  Then 

min  Ciir.w.m ) > max  T t — Kw. 

'•I*1  <r.j)fy„  " 

Hence,  the  lower  bound  LB(o-)  for  all  possible  permutations  <r  ...  is 
(9)  LB(<t)  = max  [f(or,w)  + Kw], 

k+^ 

If  a — <t>  {k  — 0)  then  (9)  provides  a lower  bound  for  T(P,m). 


EXAMPLES: 

1 . Let  y w is  a RD  segment.  Then  (see  Case  a) 

m- 1 n im  — \ n 

Kw  - Z'*  + I '«  = t min  + X ‘rm 

s — w r— A:  + l (s-H’  r— /t  + l 

2.  If  y sub  w is  a RDR  segment  then  (see  Case  b') 


K 


W 


max 


h- 1 * 

Z*« + Z ^ + 


r-*  + l 


where 


h — \ n 

Zr« + z **  + 


s-h>  r-*  + l 


T r/s  = min 
s-Tti 


Z'/«  + Z + Z * 

s-H-  r-Ar+l  s-/i+l 


fc+1 


3.  Assume  that  yK  is  a DRD  type.  According  to  Case  b,  the  optimal  solution  ir.of  the  AB 
problem  is  Ir+1 n. 


K 


W 


max 

k + 1 


q m- 1 n 

I L+  Z ffS  + Z (rm 

r-A+1  s-w  + 1 r-<y 


m- 1 


Observe  that  - l(ir  J - £ £ tn  where  r(rr  J is  the  (minimum)  processing  time  of 

r— * + 1 s-h»  + 1 

sequence  tt*  of  the  two  machine  AB  problem.  Consequently, 


LB  (a)  — max 

* + 1 < M><  * 


m-l 


Tfo-.w)  + /(tt  J - £ £ fn 

r-*+l  j-w+1 


Define  LBX  as  LB  (a)  where  x indicates  the  type  of  y »,.  In  view  of  Remark  2 and  the  definition 
of  Kw,  the  following  inequalities  hold  for  any  <r: 


ma x(LBa,LBa)  < min (LBb,LBb), 
ma x(LBb,LBb)  < min (LBC,LBC),  and 
ma x{LBc,LBc)  < LBd. 


(10) 
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ABSTRACT 

The  0-1  multiple-knapsack  problem  is  an  extension  of  the  well-known  0-1 
knapsack  problem.  It  is  a problem  of  assigning  m objects,  each  having  a value 
and  a weight,  to  n knapsacks  in  such  a way  that  the  total  weight  in  each  knap- 
sack is  less  than  its  capacity  limit  and  the  total  value  in  the  knapsacks  is  maxim- 
ized. 

A branch-and-bound  algorithm  for  solving  the  problem  is  developed  and 
tested.  Branching  rules  that  avoid  the  search  of  redundant  partial  solutions  are 
used  in  the  algorithm.  Various  bounding  techniques,  including  Lagrangean  and 
surrogate  relaxations,  are  investigated  and  compared. 


1.  INTRODUCTION 

The  0-1  knapsack  problem  is  a familiar  one  in  operations  research.  In  its  simplest  form, 
the  problem  is  to  find  the  most  desirable  set  of  objects  to  place  in  a single  container  of  given 
capacity.  Eilon  and  Christofides  [4]  describe  a series  of  generalizations  to  the  0-1  knapsack 
problem,  each  of  which  they  define  as  a type  of  0-1  loading  problem.  The  type  of  0-1  loading 
problem  we  address  in  this  paper  may  be  formulated  as  follows: 

(1)  (P)maximize  z — 11^ 

i J 

(2)  subject  £ w,x,i  < 6,  for  all  j, 

I 

(3)  £ x,7  ^ 1 for  all  /, 

J 

(4)  Xjj  - 0, 1 for  all  i.j, 

where  c,  and  w,  represent  the  value  and  weight,  respectively,  of  each  of  m objects  and  b, 
represents  the  capacity  constraint  on  each  of  n containers.  Our  objective  is  to  choose  objects 
for  placement  into  containers  so  as  to  maximize  the  total  value  of  the  objects  chosen  without 
violating  any  of  the  capacity  constraints  bt.  We  assume  that  the  objects  are  indexed  such  that 
fi/w|  > c2/w2  ^ . . . . > cm/wm.  We  define  this  particular  type  of  0-1  loading  problem  as  the 
0-1  multiple-knapsack  problem. 
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A great  deal  of  research  has  been  done  on  various  forms  and  generalizations  of  the  0-1 
knapsack  problem  (see  Salkin  and  de  Kluyver  [18]  for  a survey  of  the  knapsack  problem  and 
related  problems).  Little  work  has  been  done  on  (P),  however.  Eilon  and  Christofides  discuss 
the  application  of  problem  forms  such  as  (P)  but  do  not  suggest  a solution.  In  a more  recent 
paper,  Ingargiola  and  Korsh  [15]  describe  a series  of  rules  useful  for  limiting  the  search  for 
optimality,  though  little  computational  experience  is  given  to  indicate  the  effectiveness  of  these 
rules. 

In  this  paper,  we  wilt  present  a brancn-and-bound  procedure  for  solving  (P).  Emphasis  is 
on  an  empirical  computational  study  of  various  new  bounding  techniques,  including  Lagrangean 
relaxation  and  surrogate  relaxation.  Section  2 discusses  the  relaxations  and  the  solution  of  their 
multipliers.  Section  3 presents  the  algorithm  in  detail.  After  a general  description,  two  subsec- 
tions discuss  more  difficult  parts  of  the  algorithm.  Section  4 presents  computational  results  that 
indicate  the  relative  efficiency  of  each  of  the  relaxation  procedures  presented  in  Section  2. 

2.  RELAXATIONS  OF  THE  MULTIPLE-KNAPSACK  PROBLEM 

In  this  section  we  discuss  two  relaxation  methods  found  to  be  useful  in  providing  tight 
bounds  for  discrete  optimization  problems,  the  Lagrangean  and  the  surrogate  relaxations. 

We  define  the  Lagrangean  relaxation  of  (P)  relative  to  a nonnegative  vector  X—(X,)  to  be 

(5)  (PRJ  maximize  £ X c<xn  ~ X MX  xu  ~ D 

< i i J 

- X(c-  - *,)  Xx</+  X x< 

i J i 

(6)  subject  to  £ w,  bf  for  all  j, 

i 

(7)  x„  - 0, 1 for  all  i,j. 

An  important  feature  of  the  relaxed  problem  (PR*)  is  that,  once  values  of  X — (X,)  are  found, 
it  decomposes  into  a collection  of  single-knapsack  problems,  one  for  each  j.  Ross  and  Soland 
[17]  used  a similar  approach  to  solve  the  generalized  problem. 

Another  potentially  useful  relaxation  of  (P)  is  surrogate  relaxation.  Unlike  the 
Lagrangean  solution  strategy,  which  absorbs  a set  of  constraints  into  the  objective  function,  this 
strategy  replaces  the  original  set  of  constraints  by  a new  one  called  a surrogate  constraint.  The 
concept  and  the  applicability  of  the  surrogate  procedure  were  introduced  by  Glover  [10],  [II], 
and  useful  refinements  of  the  procedure  were  suggested  by  Balas  [2],  and  Geoffrion  [6],  Addi- 
tional work  in  this  area  has  been  presented  by  Greenberg  [12]  and  Greenberg  and  Pierskalla 
[13],  among  others. 

The  surrogate  relaxation  of  (P)  can  be  defined  as 


(8) 

(PR*)  maximize 

X X c>x'< 

i j 

(9) 

subject  to 

IXffWj<  Xff/*A 

J i i 

(10) 

< 1,  for  all  i 

J 

(11) 

x,/  — 0,1  for  all  ij. 
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Surrogate  relaxation  converts  problem  (P)  into  a single- knapsack  problem.  This  can  be 
observed  if  we  replace  with  y,  in  (8)  through  (10),  thus  reducing  (PRJ  to 

j 


(12) 

(PR„)  maximize 

Ec,y, 

i 

(13) 

subject  to 

J i 

(14) 

y,  — 0, 1 for  all  /. 

It  has  been  pointed  out  (Geoffrion  [7],  Greenberg  and  Pierskalla  113])  that  such  relaxa- 
tions can  provide  a bound  closer  to  the  optimal  value  of  the  integer  solution  than  does  the 
linear  programming  relaxation.  The  tightness  of  the  bound  clearly  depends  on  the  choice  of  the 
multipliers;  i.e.,  A for  (PRX)  and  it  for  (PR„).  One  suitable  choice  is  to  set  them  equal  to  the 
optimal  dual  multipliers  of  the  continuous  (linear  programming)  problem  of  (P) . We  see  in  the 
following  paragraphs  that  these  dual  multipliers  are  easily  found. 

Let  (P)  denote  the  continuous  relaxation  of  (P)  by  replacing  (4)  with  1 > xtJ  > 0 for  all 
i,  j.  An  optimal  solution  to  (P)  can  be  readily  found  because  (P)  contains  the  properties  that 
Dantzig  [3]  identified  in  the  0-1  single-knapsack  problem.  Let  (x;)  denote  optimal  solution  to 
(P).  Since  the  objects  are  ordered  such  that  Ci/w1>c2/h’2>.  . . , we  obtain  the  fol- 

lowing results  for  (xy): 

1 if  i < t 

JE,  J w,  if*  - » 

0 if  i > t 

where  t is  the  smallest  object  index  such  that 
(17) 

i<r  i 


(16) 

J 


_Let  (A,)  and  ( it)  be,  respectively,  the  optimal  dual  multipliers  of  constraints  (2)  and  (3) 
in  (P).  By  the  complementary  slackness  theorem 

{Cj—  w,(cjw)  if  i<t, 

0 if  i>t, 


and 


(19)  it  j - cj  w,  for  all  j. 


The  term  (PRX)  denotes  the  Lagrangean  problem  when  A — (A,)  is  used  as  the  multi- 
plier vector.  As  noted  before,  (PRX)  decomposes  into  a series  of  single-knapsack  problems,  one 
for  each  j.  Similarly,  when  (if)  are  used  in  the  surrogate  problem  (PR„)  to  obtain  (PR;),  a 
single-knapsack  problem  is  defined  as 


(20) 

(PR^max 

'Ley, 

(21) 

subject  to 

L Ifil 

1 J 

(22) 

>',  — 0,1  for  all 
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Although  other  multipliers  may  provide  better  bounds  than  X and  n,  we  feel  that,  since  X 
and  Tf  are  so  easy  to  compute,  the  extra  computional  effort  required  to  find  other  multipliers 
would  overshadow  the  savings  realized.  It  is  important  to  note  that  the  bounds  derived  from 
(PR;;)  or  (PRS)  are  always  better  than  or  equal  to  the  bounds  from  linear  programming  relaxa- 
tion. 

3.  The  Branch-and-Bound  Algorithm 

The  essential  steps  of  our  algorithm  are  closely  related  to  those  developed  for  the  0-1 
single-knapsack  problem  by  Ahrens  and  Finke  [1]  among  others  [5,16].  The  general  form  of 
these  algorithms  can  best  be  described  as  being  of  a "branch  and  exclude"  type  which  succes- 
sively determines  whether  or  not  an  object  is  to  be  in  the  knapsack.  The  presence  of  multiple 
knapsacks  in  our  problem  demands  a further  decision:  To  what  knapsack  is  the  object  to  be 
assigned  next? 


In  this  paper,  we  construct  successively  higher  levels  of  the  branch-and-bound  tree  either 
by  assigning  an  object  to  a knapsack  or  by  excluding  that  object  from  all  knapsacks.  Any 
chosen  level  of  our  branch-and-bound  tree,  therefore,  represents  the  assignment  of  some 
objects  to  the  knapsacks  and  the  exclusion  of  some  other  objects  from  all  knapsacks.  For  sim- 
plicity, assume  that  the  excluded  objects  have  been  assigned  to  the  (n-l-l)”  (dummy)  knapsack. 
Let  F denote  the  index  set  of  the  objects  that  have  not  been  assigned.  When  F=<i>,  we  have 
found  a feasible  solution  and  the  corresponding  objective  value  z,  which  can  then  be  compared 
to  the  incumbent  solution  value  z * 


Define  S to  be  the  index  set  of  the  objects  that  have  been  assigned:  S 


n+1 


J- 1 


1 


If  M is  the  index  set  of  all  objects,  then  S U F = M and  S O F = <t>.  The  cardinality  of  set  5 
thus  coincides  with  the  level  of  the  branching  tree.  To  avoid  excessive  notation,  we  simply 
note  that  if  S r*  <f>,  then  (P)  and  its  relaxed  problems  (PRX)  and  (PR„)  are  reduced  to  prob- 
lems with  respect  to  objects  in  Fonly. 


A general  description  of  the  steps  of  our  algorithm  is  as  follows: 

STEP  1 (Initialization):  Set  z ' = 5 — 0,  F = M.  Let  tree-level  index  k = 1. 


STEP  2 (Bounding):  Solve  the  relaxed  problem  (PRX)  or  (PRjP  with  respect  to  F.  Com- 
pute the  value  of  such  solution,  zk,  by  utilizing  the  original  objective  function  (1).  If  z*  < z*, 
go  to  Step  5.  If  the  solution  is  feasible  to  (P),  go  to  Step  4;  otherwise,  go  to  Step  3. 

STEP  3 (Branching):  Select  an  object  / and  assign  the  object  to  one  of  the  knapsacks 
(including  the  dummy  knapsack).  Record  the  value  of  the  objects  in  the  knapsacks  (excluding 
the  dummy).  If  all  objects  have  been  assigned,  go  to  Step  4,  otherwise  update  S,/7,  and  k and 
go  to  Step  3. 

STEP  4:  Update  z*  and  its  corresponding  solution.  Go  to  Step  5. 

STEP  5:  (Backtracking):  F;nd  the  smallest  level  k such  that  z*0<z*.  Denote  the  level 
preceding  k 0 as  (i.e.,  k_|  — k0—  1),  and  denote  the  corresponding  object  index  in  S as  ik  _j. 
If  k_i<0,  terminate  the  procedure,  otherwise  set  k — and  free  all  indices  in  S following 
ik  r Finally,  assign  object  ik  ( to  a knapsack  that  is  different  from  the  knapsacks  to  which  it  has 
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previously  been  assigned  and  go  to  Step  2.  If  object  /*_|  has  been  previously  assigned  to  each 
of  the  /i+l  boxes,  however,  set  z*  ( — -oo  and  go  to  Step  5. 

Detailed  descriptions  of  steps  2,3,  and  5 above  are  provided  in  the  following  two 
subsections. 

3a:  Solving  the  relaxed  problems  (Step  2) 

Both  the  Lagrangean  relaxation  and  the  surrogate  relaxation  reduce  the  original  problem 
(P)  to  single-knapsack  problems.  Our  single-knapsack  algorithm  is  adopted  from  program  /3  of 
Ahrens  and  Finke  II).  The  algorithm  is  easy  to  use  and  has  performed  satisfactorily. 

A few  rules  were  added  to  the  single-knapsack  algorithm  to  increase  efficiency.  At  a given 
level  of  our  branching  tree,  some  of  the  knapsacks  have  been  assigned  objects.  For  each  knap- 
sack j,  let  fj  be  its  unassigned  capacity,  that  is 

(23)  /,  - b,-  £ w,Xii  for  all  j. 

itS 

RULE  1:  Knapsack  j is  deleted  from  (PR*)  and  (PR  J if  /,  < rninw,,  J - 1 n. 

i*F 

RULE  2:  Object  / is  deleted  from  (PRW)  if  h>,  > max/,  /eF 

i 

RULE  3:  For  each  single  knapsack  j in  (PR*),  object  / is  deleted  from  consideration  if 

*,>/,,  icf. 

We  have  found  such  simple  rules  to  be  very  effective  in  reducing  the  amount  of  computa- 
tion. 

3b.  Assigning  objects  to  knapsacks  (Steps  3 and  5) 

In  considering  Steps  3 and  5,  we  must  make  two  important  decisions:  (1)  which  object  to 
choose  for  branching  (Step  3)  and  (2)  to  which  knapsack  should  an  object  be  assigned  (Steps  3 
and  5). 

The  branching  object  we  choose  depends  upon  the  type  of  relaxation  employed.  In  the 
case  of  Lagrangean  relaxation,  we  choose  that  object  in  F which  appears  in  the  most  single- 
knapsack solutions  involved  in  calculating  (PR*)-  In  the  event  of  ties,  the  lowest-indexed 
object  in  Fis  chosen.  This  choice  of  branching  object  has  two  advantages.  First,  it  has  the  abil- 
ity to  change  the  most- violated  constraint  in  (3)  of  the  original  problem  into  a satisfied  con- 
straint. Second,  it  will  generally  generate  a tighter  upper  bound  for  use  at  the  next  level  of  the 
branching  tree. 

Since  the  surrogate  relaxation  does  not  solve  a series  of  single-knapsack  problems  to  cal- 
culate a bound,  the  object  in  F which  we  choose  to  branch  on  is  simply  the  lowest-indexed  one. 
The  object  chosen  by  this  rule  represents  the  object  that  has  the  highest  value-to-weight  ratio 
among  objects  not  yet  assigned  and  thus  has  the  highest  a priori  chance  of  being  in  the  optimal 
solution  of  (P). 
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When  considering  the  assignment  of  a chosen  object  to  a knapsack,  we  use  several  rules 
developed  by  Hung  and  Brown  [14]  for  this  type  of  assignment.  The  rules  are  useful  to  reduce 
the  number  of  knapsacks  an  object  may  be  assigned  to. 

Assume  that  objects  of  the  same  weight  belong  to  a class  and  that  knapsacks  of  the  same 
capacity  belong  to  a class.  The  order  of  objects  in  a class  is  determined  by  the  order  in  which 
they  are  chosen  as  branching  objects.  (This  ordering  does  not  change  the  indexing  determined 
by  value-to-weight  ratios  at  the  beginning  of  the  algorithm.)  However,  we  assume  that  knap- 
sacks are  indexed  in  descending  order  of  their  capacity,  i.e.,  ^ bn . Therefore, 

knapsacks  of  the  same  capacity  class  are  grouped  together.  For  simplicity  of  expression,  by  a 
"preceding  object"  to  object  / we  mean  an  object  in  the  same  class  as  object  / whose  order  in  the 
class  precedes  object  /. 

The  rules  for  assigning  an  object  / to  knapsacks  are 

RULE  4:  Object  / can  be  assigned  to  knapsack  j if  either  (a)  object  / is  the  first  object  in 
its  class  to  be  considered  or  (b)  the  preceding  object  was  assigned  to  a knapsack  whose  index  is 
not  greater  than  J. 

RULE  5:  Object  / can  be  assigned  to  knapsack  j if  either  (a)  j is  the  smallest  index  in  its 
class  or  (b)  knapsack  J— 1 is  not  empty. 

RULE  6:  Object  / can  be  assigned  to  knapsack  j if  w,</„  where  /,  is  the  remaining  capa- 
city of  knapsack  j as  defined  in  (21). 

For  ease  of  bookkeeping  and  backtracking,  we  define  n,  to  be  an  index  set  of  all  knap- 
sacks to  which  object  / can  be  assigned.  Note  that  if  an  object  cannot  be  assigned  to  any  knap- 
sack j,  1 <_/<n,  according  to  Rules  4 through  6 we  let  n,  contain  the  index  of  the  dummy 
knapsack  only.  Once  n,  is  determined,  we  assign  object  / to  the  knapsack  which  has  the  smal- 
lest index  in  n,  and  then  remove  the  index  from  /*,.  When  we  return  to  object  / during  the 
backtracking  step  (Step  5),  we  simply  reassign  object  / to  the  lowest-indexed  knapsack  remain- 
ing in  n,. 

For  a detailed  discussion  of  the  branching  rules  4 through  6,  the  reader  is  referred  to 
Hung  and  Brown  (14). 

4,  COMPUTATIONAL  EXPERIENCE 

The  algorithm  as  described  has  been  programmed  in  FORTRAN  V code  and  run  on  a 
UNIVAC  1108.  We  obtained  series  of  problems  consisting  of  up  to  200  objects  and  up  to  six 
boxes  by  generating  values  and  weights  independently  from  a uniform  distribution  in  the  inter- 
val [10,100].  Box  capacities  were  then  generated  in  a similar  manner,  except  that  the  interval 
bt  O,  Kb„  was  used,  where  6,  - [0.4(Ih>,  /n)]  and  bu  - [0.6(£w,  //»)]*.  The  final  box  capa- 
city generated,  b„ , was  chosen  such  that  occupancy  ratio  — I b,  /I w,  - 0.5.  If  b , < min  w„  or 

i 

max  b i < max  w„  the  set  of  generated  box  capacities  was  discarded  and  a new  set  was  gen- 

/ < 

erated.  The  occupy  ,vy  ratio  of  0.5  was  used  for  all  problems  attempted. 

Tables  1 and  2 indicate  computation  times  for  our  algorithm,  with  first  the  surrogate 
relaxation  (Table  1)  then  the  Lagrangean  relaxation  (Table  2),  to  determine  upper  bounds  on 


*The  brackets  | 1 denote  the  greatest  integer  less  than  the  enclosed  quantity. 
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TABLE  1 — Surrogate  Relaxation 


Number 

of 

Objects 

Two  Boxes 

Three  Boxes 

Four  Boxes 

Complete 

Solutions 

Solution  Time  (S) 

Complete 

Solutions 

Solution  Time  CS) 

Complete 

Solutions 

Solution  Time  (S) 

High 

Low 

Av. 

High 

Low 

Av. 

High 

Low 

Av. 

20 

10 

0.2 

0.01 

0 1 

10 

4.1 

0.01 

0.4 

10 

33.6 

0.3 

9.6 

30 

10 

0.20 

0.01 

0.1 

10 

S.4 

0.03 

0.8 

10 

75.9 

0.3 

22.3 

40 

10 

0.6 

0.04 

0.2 

10 

52.5 

0.4 

19.6 

1 

_ 



60 

10 

2.0 

0.1 

0.7 

1 

— 

— 

— 

1 

— 

_ 

_ 

80 

3 

— 

— 

— 

0 

_ 

_ 

0 

_ 

_ 

100 

0 

- 

- 

0 

- 

- 

- 

0 

- 

- 

- 

TABLE  2 — Lagrangean  Relaxation 


Number 

of 

Objects 

Two  Boxes 

Three  Boxes 

Four  Boxes 

Complete 

Solutions 

Solution  Time  (S) 

Complete 

Solutions 

Solution  Time  (S) 

Complete 

Solutions 

Solution  Time  (S) 

High 

Low 

Av. 

High 

Low 

Av. 

High 

Low 

Av. 

20 

10 

1.5 

0 1 

0.4 

10 

35.1 

0.2 

40 

10 

90.1 

08 

18  6 

30 

10 

5.6 

0.3 

1.1 

10 

3.7 

0.3 

09 

10 

83.3 

0.4 

10.9 

40 

10 

5.1 

0.8 

2.3 

10 

80.2 

0.7 

16.1 

4 





60 

10 

7.4 

1.0 

2.8 

10 

38.1 

1.7 

68 

7 

_ 

_ 



80 

10 

21.0 

2.2 

7.2 

10 

14.2 

2.7 

66 

5 







100 

10 

11.0 

39 

65 

10 

9.7 

4.3 

7.1 

4 

- 

- 

- 

partial  solutions.  The  sets  of  rules  defined  in  Sections  3a  and  3b  were  used  in  each  case.  For 
each  object/box  combination  a total  of  ten  problems  was  attempted,  and  the  total  number  of 
complete  solutions  obtained  within  a maximum  of  250-s  running  time,  excluding  input/output 
time,  is  specified.  For  those  problem  sets  requiring  < 250-s  for  completion  of  ten  problems, 
the  maximum,  minimum,  and  average  solution  times  are  also  specified.  Due  to  the  method  in 
which  problem  sets  were  generated,  each  relaxation  procedure  attempted  to  solve  exactly  the 
same  set  of  problems. 

The  results  summarized  in  Tables  1 and  2 indicate  that  the  surrogate  relaxation  is  rela- 
tively more  efficient  when  the  number  of  objects  and/or  boxes  is  small,  but  that  the  Lagrangean 
relaxation  is  vastly  superior  when  one  attempts  to  solve  larger  problems.  Also,  our  experience, 
as  shown  in  Tables  1 and  2,  would  indicate  that,  although  the  number  of  objects  is  an  important 
factor  in  estimating  solution  time,  the  number  of  boxes  has  a tremendous  influence ".  A further 
indication  of  the  effect  of  the  number  of  boxes  can  be  shown  by  the  results  obtained  in  the 
attempt  to  solve  an  additional  set  of  ten  problems  consisting  of  20  objects  and  six  boxes  (occu- 
pancy ratio  - 0.5).  The  computer  ran  out  of  time  after  having  solved  only  3 of  these  prob- 
lems. 


In  an  attempt  to  improve  upon  the  solution  times  presented  in  Tables  1 and  2,  we  solved 
a series  of  problems  in  which  we  applied  both  the  surrogate  relaxation  and  the  Lagrangean 
relaxation  for  each  candidate  problem  then  used  the  bound  having  minimum  value  as  a test 
against  the  incumbent.  While  the  number  of  branches  in  the  branch-and-bound  tree  was  in 
general  reduced  with  this  method,  the  results  were  not  significantly  better  and  are  not  reported 
here. 


*Thi*  should  not  be  surprising,  sines  the  number  of  distinct  solutions  to  (P)  can  be  characterized  by  (n+l)m  For  a 
20-object  problem,  then,  the  maximum  number  of  solutions,  if  we  use  only  one  box,  is  1.04  x 10*.  but  for  six  boxes 
this  number  increases  to  more  than  7.9  x |014 
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As  has  been  shown  in  Tables  1 and  2,  it  becomes  increasingly  difficult  to  obtain  optimal 
answers  to  (P)  when  the  problem  size  becomes  large.  We  were,  therefore,  curious  to  see  how 
efficient  our  Lagrangean  relaxation  routine  would  be  if  we  were  to  require  solutions  which  were 
within  some  small  percentage  value  a of  a known  upper  bound  on  the  optimal  solution  to  (P). 
Since  each  of  the  initial  surrogate  and  Lagrangean  relaxations  yields  such  an  upper  bound,  we 
determined  to  specify  a - 0.5%  and  to  run  a series  of  problems  generated  in  the  usual  manner 
but  to  suspend  calculations  when  we  found  a solution  value  o/lOO),  where  R 

represents  the  minimum  value  associated  with  the  initial  surrogate  and  Lagrangean  relaxations. 
The  results  of  these  calculations  are  shown  in  Table  3. 

As  can  be  seen  from  Table  3,  our  Lagrangean  relaxation  technique  becomes  very  efficient 
when  we  require  only  that  our  solutions  lie  within  some  small  value  of  the  known  upper  bound. 
These  results  are  very  encouraging  when  they  are  considered  in  the  light  of  the  obvious 
difficulty  in  obtaining  provably  optimal  solutions  to  large  problems. 


TABLE  3 — Lagrangean  Relaxation  (a  — 0.5%) 


In  comparing  our  results  with  previous  work,  the  only  other  computational  experience  we 
found  was  that  of  Ingargiola  and  Korsh  [15]  in  which  they  solve  a set  of  ten  problems  consist- 
ing of  15  objects  and  6 knapsacks  (occupancy  ratio  * 0.2  ).  Their  average  computation  time 
for  this  set  of  problems  was  7 seconds  on  an  IBM  370-155.  We  were  unable  to  generate  ran- 
dom problems  having  the  characteristics  defined  by  Ingargiola  and  Korsh,  due  to  our  restriction 
that  all  item  weights  be  less  than  or  equal  to  the  largest  box  capacity.  When  we  suspended  this 
restriction,  our  Lagrangean  relaxation  procedure  required  an  average  of  0.2  seconds  per  prob- 
lem. 
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